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Foreword 


When I first became seriously interested in enumerative combinatorics around 1967, 
the subject did not really exist per se. There were numerous results and methods, 
beginning with Euler (with some hints of even earlier work), scattered throughout the 
literature, but there was little systematic attempt to bring some order to this chaos. 
It is remarkable that enumerative combinatorics has progressed so far that we now 
need over 1000 pages just to present a basic overview of techniques and results. 
Numerous topics such as pattern avoidance and parking functions existed in only 
very rudimentary form 40-50 years ago but are now flourishing subjects in their own 
right. The seventeen authors of the present volume, in addition to being world leaders 
in the area of their contribution, are also superb writers. Their fifteen chapters are a 
combination of broad surveys of major areas and techniques, together with more 
specialized expositions of many of the most active research topics in enumerative 
combinatorics today. 

A prominent reason why practitioners of enumerative combinatorics find it so ap- 
pealing is its unexpected connections with other areas of mathematics. These connec- 
tions have grown increasingly sophisticated over the years. It is no longer sufficient 
to know some rudimentary algebraic topology, say, to give a significant connection 
with enumerative combinatorics. The papers in this handbook do an exemplary job 
of explaining deep connections with such areas as complex analysis, probability the- 
ory, linear algebra, commutative algebra, representation theory, algebraic geometry, 
algebraic topology, and computer science. Just this list of topics gives an idea of the 
breadth and depth of modern enumerative combinatorics. Readers from neophytes to 
experts have much to look forward to when they peruse the riches that follow. 


Richard Stanley 
Cambridge, MA 
December 2014 
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Preface 


When designing a handbook of a large and rapidly developing fields like Enumerative 
Combinatorics, one faces several questions: What subjects to cover? How to organize 
the subjects? What audience to target? 

We have decided to include both chapters that focus on methods of enumeration 
of various objects and chapters that focus on specific kinds of objects that need to 
be counted, by any method available. The chapters are organized so that we advance 
from the more general ones, namely enumeration methods, towards the more special- 
ized ones that focus on the counting of specific objects. These objects become more 
and more specialized as we proceed. 

As far as our preferred audience goes, we believe that each chapter can benefit at 
least three different kinds of readers as listed below. 


e The experts, who are familiar with most of the information in the chapter, but 
are interested in its presentation, and some of the finer points. 


e The “relative outsiders,” that is, readers who have already seen a few results 
here and there, a proof or two here and there, but nothing systematic, and 
who are interested to see an organized treatment of the topic. 


e The novices, who are new to the field, and have no background information 
past the first year of graduate school. These readers will hopefully see that 
the subject is interesting, accessible, and challenging. 


We do hope that all three groups of our targeted readership will find the book as 
useful and enjoyable as we do. 


Miklos Bona 


Gainesville, FL 
February 2015 


XXi 


Acknowledgments 


At the completion of this book, I want to say thanks to Robert Ross at CRC Press, 
who initiated the project and selected me to lead it. The most important part of this 
endeavor were certainly the authors, who agreed to the difficult and time-consuming 
task of contributing a chapter to the book, and fulfilled that task, sometimes on time. 
My gratitude is extended to the mathematicians who have reviewed the various chap- 
ters. Turning 15 chapters from 17 different authors into one consistent LaTeX file was 
often too difficult for me, so I am grateful to Shashi Kumar at cenveo.com, who often 
helped me with technical matters. My gratitude is extended to my colleague Vincent 
Vatter, who was my local expert. 

A non-negligible part of my editing work was done while I was on vacation. I 
owe a lot to my brother Péter who made sure that I had high-speed internet access 
when most people around us could not even make a phone call. Last, but not least, I 
am thankful to my wife Linda, and my sons Miki, Benny, and Vinnie for putting up 
with me when I worked at the book at highly unexpected times. 


XXill 


Part I 


Methods 


Chapter | 


Algebraic and Geometric Methods 
in Enumerative Combinatorics 


Federico Ardila 
San Francisco State University, San Francisco, CA, USA; Universidad de los Andes, 
Bogotd, Colombia 


CONTENTS 
1.1 Introduction 
1.2 
1.3 Generating functions 
1.3.1 
1.3.2 
1.3.2.1 
1.3.2.2 
1.3.3 
1.3.3.1 
1.3.3.2 
1.3.4 
1.3.4.1 
1.3.4.2 
1.4 


What is a good answer? ........ ee ccc nee eee e tebe enes 


The ring of formal power series ............. 0.0 c eee e eee 
Ordinary generating functions .......................0.008. 


Operations on combinatorial structures and 
their generating functions .................... 
Examples:c 00 faseiaves eds tanasosiead ceenns 


Exponential generating functions .................... eee ee 


Operations on labeled structures and their 
exponential generating functions ............. 
Bxamples:3iccsteiatanut see ye heaia beds 


Nice families of generating functions ....................04. 


Rational generating functions ................ 
Algebraic and D-finite generating functions ... 


Linear algebra methods ............. ccc cece cece e eee een n ene nes 


1.4.1 


1.4.2 


Determinants in combinatorics ............ 0. cece eee ees 


1.4.1.1 
1.4.1.2 
1.4.1.3 
1.4.1.4 
1.4.1.5 
1.4.1.6 


Preliminaries: Graph matrices ................ 
Walks: the transfer matrix method ............ 
Spanning trees: the matrix-tree theorem ...... 
Eulerian cycles: the BEST theorem ........... 
Perfect matchings: the Pfaffian method ....... 
Routings: the Lindstr6m—Gessel—Viennot 

J€MM A’ 204 feceas ces ewes kde ase eee ws eee 


Computing determinants ....................0 0... c eee eee 


1.4.2.1 


Is it known? 20... 0... 


15 


1.6 


1.7 


1.8 


Handbook of Enumerative Combinatorics 


1.4.2.2 Row and column operations .................. 66 
1.4.2.3 Identifying linear factors ..................04. 66 
1.4.2.4 Computing the eigenvalues ................... 67 
1.4.2.5 LU factorizations ...................0 00.0008. 68 
1.4.2.6 Hankel determinants and continued fractions . 68 
POSCtS) ced aoc sav sve saccea anes tuba laevaughiasels foie wena needy 71 
1.5.1 Basic definitions and examples .......................0.008. 72 
V52e 5 < LeattiGes sets ee CS eee IE NR 74 
1.5.3. Zeta polynomials and order polynomials ................... 77 
1.5.4 The inclusion-exclusion formula .....................0.004. 719 
1.5.5. Mobius functions and Mobius inversion .................... 81 
1.5.5.1 The Mobius function ................0....005. 81 
15.522 MObius inversion ...................0.0 2c eee, 83 
1.5.5.3 The incidence algebra ....................004. 85 
1.5.5.4 Computing Mobius functions ................ 86 
1.5.6 Eulerian posets, flag f-vectors, and flag h-vectors ........... 91 
POlytOpes: 226 tok teehee cl a comes ake a eee Lea 93 
1.6.1. Basic definitions and constructions ...................0.008. 93 
1.6:27 > Examples 0.623442 cities tence bute gaieeehs wane Rameadee dacs 98 
16:3" | “(COUNDNE faCeS: Gi. bajo saver bane talaga anda nie Lagu ate 100 
1.6.4 Counting lattice points: Ehrhart theory ..................... 102 
Hyperplane arrangements .............. 0... cece cece cece eee 108 
V7: Basi¢e- definitions... 2:5 tics ceccc ape etecuaeteia ces aria eedecete 109 
1.7.2 The characteristic polynomial ........................0.0008. 110 
1.7.3. Properties of the characteristic polynomial ................. 112 
1.7.3.1 Deletion and contraction ..................04. 112 
1.7.3.2 Sign alternation and unimodality ............. 114 
1.7.3.3 Graphs and proper colorings ................. 114 
1.7.3.4 Free arrangements .................0.000005. 115 
1.7.3.5 Supersolvability ....................0....008. 116 
1.7.4 Computing the characteristic polynomial ................... 117 
1.7.5 The cd-index of an arrangement .....................0.004. 125 
Mattoids sistent dei de cdi toodis oiteaatebe abide cea edamende 126 
1.8.1 Main example: Vector configurations and linear matroids .... 127 
1.8.2 Basic definitions ............... 0. 128 
1.8.3. Examples. oiccsicib cidesosiee ehh aa deedede see cideade dade 130 
1.8.4 Basic constructions ............ 0... 133 
1.8.5 A few structural results ............... 00. c eee eee 134 
1.8.6 The Tutte polynomial ................... 02 eee 138 
1.8.6.1 Explicit definition ..................0....005. 138 
1.8.6.2 Recursive definition and universality property . 138 
1.8.6.3 Activity interpretation ....................04. 140 
1.8.6.4 Finite field interpretation ..................... 140 
1.8.7. Tutte polynomial evaluations ........................0.008. 141 


1.8.7.1 General evaluations ................c cece eee ee 142 


Algebraic and Geometric Methods in Enumerative Combinatorics 5 


1.8.7.2 Graphs. ids sso% heb deedagiede dees diadidedeeed 143 

1.8.7.3 Hyperplane arrangements .................... 146 

1.8.7.4 Algebras from arrangements ................. 146 

1.8.7.5 Error-correcting codes ..............0.0..005. 147 

1.8.7.6 Probability and statistical mechanics ......... 148 

1.8.7.7 Other applications .......................005. 149 

1.8.8 Computing the Tutte polynomial .....................0.00.. 150 

1.8.9 Generalizations of the Tutte polynomial .................... 153 
1.8.10 Matroid subdivisions, valuations, and the Derksen-Fink 

IDVATLANE ac acccecre abies ed saa dehes dhs ewdaeabcae eu: 155 

References Yas. dwniia. ita ats cds teow adentewen whe haa streamer tbe 158 


1.1. Introduction 


Enumerative combinatorics is about counting. The typical question is to find the 
number of objects with a given set of properties. 

However, enumerative combinatorics is not just about counting. In “real life,” 
when we talk about counting, we imagine lining up a set of objects and counting 
them off: 1,2,3,.... However, families of combinatorial objects do not come to us in 
a natural linear order. To give a very simple example: We do not count the squares in 
an m x n rectangular grid linearly. Instead, we use the rectangular structure to under- 
stand that the number of squares is m-n. Similarly, to enumerate a more complicated 
combinatorial set, we usually spend most of our efforts understanding the underlying 
structure of the individual objects, or of the set itself. 

Many combinatorial objects of interest have a rich and interesting algebraic or 
geometric structure, which often becomes a very powerful tool toward their enumer- 
ation. In fact, there are many families of objects that we only know how to count 
using these tools. Our goal in this chapter is to highlight some key aspects of the rich 
interplay between algebra, discrete geometry, and combinatorics, with an eye toward 
enumeration. 


About this chapter. Over the last fifty years, combinatorics has undergone a radi- 
cal transformation. Not too long ago, combinatorics mostly consisted of ad hoc meth- 
ods and clever solutions to problems that were fairly isolated from the rest of math- 
ematics. It has since grown to be a central area of mathematics, largely thanks to the 
discovery of deep connections to other fields. Combinatorics has become an essential 
tool in many disciplines. Conversely, even though ingenious methods and clever new 
ideas still abound, there is now a powerful, extensive toolkit of algebraic, geometric, 
topological, and analytic techniques that can be applied to combinatorial problems. 
It is impossible to give a meaningful summary of the many facets of algebraic 
and geometric combinatorics in a writeup of this length. I found it very difficult but 
necessary to omit several beautiful, important directions. In the spirit of a Handbook 
of Enumerative Combinatorics, my guiding principle was to focus on algebraic and 
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geometric techniques that are useful toward the solution of enumerative problems. 
My main goal was to state clearly and concisely some of the most useful tools in 
algebraic and geometric enumeration, and to give many examples that quickly and 
concretely illustrate how to put these tools to use. 


PART 1. ALGEBRAIC METHODS 


The first part of this chapter focuses on algebraic methods in enumeration. In Sec- 
tion 1.2 we discuss the question, “What is a good answer to an enumerative prob- 
lem.” Generating functions are the most powerful tool to unify the different kinds 
of answers that interest us: explicit formulas, recurrences, asymptotic formulas, and 
generating functions. In Section 1.3 we develop the algebraic theory of generating 
functions. Various natural operations on combinatorial families of objects correspond 
to simple algebraic operations on their generating functions, and this allows us to 
count many families of interest. In Section 1.4 we show how many problems in com- 
binatorics can be rephrased in terms of linear algebra, and reduced to the problem 
of computing determinants. Finally, Section 1.5 is devoted to the theory of posets. 
Many combinatorial sets have a natural poset structure, and this general theory is 
very helpful in enumerating such sets. 


1.2. What is a good answer? 


The main goal of enumerative combinatorics is to count the elements of a finite set. 
Most frequently, we encounter a family of sets 7o,7),7,73,... and we need to find 
the number ¢,, = |T,,| forn = 1,2,.... What constitutes a good answer? 

Some answers are obviously good. For example, the number of subsets of 
{1,2,...,n} is 2”, and it seems clear that this is the simplest possible answer to 
this question. Sometimes an answer “is so messy and long, and so full of factorials 
and sign alternations and whatnot, that we may feel that the disease was preferable 
to the cure” [218]. Usually, the situation is somewhere in between, and it takes some 
experience to recognize a good answer. 

A combinatorial problem often has several kinds of answers. Which answer is 
better depends on what one is trying to accomplish. Perhaps this is best illustrated 
with an example. Let us count the number a, of domino tilings of a 2 x n rectangle 
into 2 x 1 rectangles. There are several different ways of answering this question. 


Explicit formula 1. We first look for an explicit combinatorial formula for a,. To 
do that, we play with a few examples, and quickly notice that these tilings are struc- 
turally very simple: They are just a sequence of 2 x | vertical tiles, and 2 x 2 blocks 
covered by two horizontal tiles. Therefore constructing a tiling is the same as writing 
nas an ordered sum of Is and 2s. For example, the tilings of Figure 1.1 correspond, 
respectively, to 1+1+1+4+1,14+1+4+2,14+2+1,2+1+1,2+2. These sums are 
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Figure 1.1 
The five domino tilings of a 2 x 4 rectangle. 


easy to count. If there are k summands equal to 2 there must be n — 2k summands 


equal to 1, and there are raae = ey) ways of ordering the summands. Therefore 


ln/2| n—k n n—1 n—2 
m= Ee G(T tC a te (1.1) 


This is a pretty good answer. It is certainly an explicit formula, and it may be used to 
compute a, directly for small values of n. It does have two drawbacks. Aesthetically, 
it is certainly not as satisfactory as “2”.” In practice, it is also not as useful as it 
seems; after computing a few examples, we will soon notice that computing binomial 
coefficients is a non-trivial task. In fact there is a more efficient method of computing 
an. 


Recurrence. Let n > 2. In a domino tiling, the leftmost column of a 2 x n can be 
covered by a vertical domino or by two horizontal dominoes. If the leftmost domino 
is vertical, the rest of the dominoes tile a 2 x (n— 1) rectangle, so there are a,_1 such 
tilings. On the other hand, if the two leftmost dominoes are horizontal, the rest of the 
dominoes tile a 2 x (n — 2) rectangle, so there are a,_2 such tilings. We obtain the 
recurrence relation 


ao=1, aqh=1, Qe =Q)-1+Q4,-2 forn>2, (1.2) 


which allows us to compute each term in the sequence in terms of the previous ones. 
We see that a, = F,,; is the (n+ 1)th Fibonacci number. 

This recursive answer is not as nice as “2”” either; it is not even an explicit for- 
mula for a,. If we want to use it to compute a,, we need to compute all the first n 
terms of the sequence 1,1,2,3,5,8, 13,21,34,55,89, 144, .... However, we can com- 
pute those very quickly; we only need to perform n— | additions. This is an extremely 
efficient method for computing dy. 


Explicit formula 2. There is a well-established method that turns linear recurrence 
relations with constant coefficients, such as (1.2), into explicit formulas. We will 
review it in Theorem 1.3.5. In this case, the method gives 


n+l n+l 
An = 4 (44) (54) (1.3) 
V5 2 2 


This is clearly the simplest possible explicit formula for a,,; in that sense it is a great 
formula. 
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A drawback is that this formula is really not very useful if we want to compute 
the exact value of, say, ajo99. It is not even clear why (1.3) produces an integer, and 
to get it to produce the correct integer would require arithmetic calculations with 
extremely high precision. 

An advantage is that, unlike (1.1), (1.3) tells us very precisely how a, grows with 
n. 


Asymptotic formula. It follows immediately from (1.3) that 
an~c-@", (1.4) 


where c = Lae and @ = Lie =~ 1.6179... is the golden ratio. This notation means 


that lim,,.0dn/(c-@") = 1. In fact, since [5% <1, dy is the closest integer to 
c-@". 


Generating function. The last kind of answer we discuss is the generating func- 
tion. This is perhaps the strangest kind of answer, but it is often the most powerful 
one. 

Consider the infinite power series A(x) = ay +. a,x +a x7 +--+. We call this the 
generating function of the sequence ag,a),a2,.... * We now compute this power 
series: From (1.2) we obtain that A(x) = 1 +x+¥,52(@n—1 + dn—2)x" = 14+x+ 
x(A(x) — 1) +x*A(x), which implies 


1 
A(x) =ap tayx+anx? 4+---= ae 
l1-x-x 


(1.5) 


With a bit of theory and some practice, we will be able to write the equation (1.5) 
immediately, with no further computations (Example 18 in Section 1.3.2). To show 
this is an excellent answer, let us use it to derive all our other answers, and more. 


e Generating functions help us obtain explicit formulas. For instance, rewriting 


J = Y¥ («t+x°)f 


A(x) = 1—(x+x2) Ea 


we recover (1.1). If, instead, we use the method of partial fractions, we get 


1//5 1/V/5 


1S, fs 12, 


A(x) = 


which brings us to our second explicit formula (1.3). 


e Generating functions help us obtain recursive formulas. In this example, we 
simply compare the coefficients of x” on both sides of the equation A(x) (1 — 
x— x) = |, and we get the recurrence relation (1.2). 


*For the moment, let us not worry about where this series converges. The issue of convergence can be 
easily avoided (as combinatorialists often do, in a way that will be explained in Section 1.3.1) or resolved 
and exploited to our advantage; let us postpone that discussion for the moment. 
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e Generating functions help us obtain asymptotic formulas. In this example, 
(1.5) leads to (1.3), which gives (1.4). In general, almost everything that we 
know about the rate of growth of combinatorial sequences comes from their 
generating functions, because analysis tells us that the asymptotic behavior 
of a, is intimately tied to the singularities of the function A(x). 


e Generating functions help us enumerate our combinatorial objects in more 
detail, and understand some of their statistical properties. For instance, say 
we want to compute the number a,,, of domino tilings of a 2 x n rectangle 
that use exactly m vertical tiles. Once we really understand (1.5) in Section 
1.3.2, we will get the answer pee 


= AmnV "xX" 


——: x2 med 


Now suppose we wish to know what fraction of the tiles is vertical in a large 
random tiling. Among all the a, domino tilings of the 2 x n rectangle, there 
are Vin>0M4mn Vertical dominoes. We compute 


Hail 1 = Xx 
Le | 2 mann |x" = Ov \1—ve—x? J |, (1x — x2)?” 


n>0 \m>0 


Partial Gactions-then tellus th n (14v5\""" 4 
artial fractions then tell us that Y,s9Mdmn ~ 3 (2 ~ FgNGn. 


Hence the fraction of vertical tiles in a random domino tiling of a 2 x n rect- 
angle converges to 1/v5 asin — ©, 


So what is a good answer to an enumerative problem? Not surprisingly, there 
is no definitive answer to this question. When we count a family of combinatorial 
objects, we look for explicit formulas, recursive formulas, asymptotic formulas, and 
generating functions. They are all useful. Generating functions are the most powerful 
framework we have to relate these different kinds of answers and, ideally, find them 
all. 


13 Generating functions 


In combinatorics, one of the most useful ways of “determining” a sequence of num- 
bers ao,a1,d2,... is to compute its ordinary generating function 


A(x) = Sax = ag tayxt+anx? +.a3x° + asx’ 4 
n>0 
or its exponential scene: function 
x2 x x4 
Agate =P aX = ay bayx tar > +43 +a4sa te 


n>0 
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This simple idea is extremely powerful because some of the most common algebraic 
operations on ordinary and exponential generating functions correspond to some of 
the most common operations on combinatorial objects. This allows us to count many 
interesting families of objects; this is the content of Section 1.3.2 (for ordinary gener- 
ating functions) and Section 1.3.3 (for exponential generating functions). In Section 
1.3.4 we see how nice generating functions can be turned into explicit, recursive, and 
asymptotic formulas for the corresponding sequences. 

Before we get to this interesting theory, we have to understand what we mean 
by power series. Section 1.3.1 provides a detailed discussion, which is probably best 
skipped the first time one encounters power series. In the meantime, let us summarize 
it in one paragraph: 

There are two main attitudes toward power series in combinatorics: the analytic 
attitude and the algebraic attitude. To harness the full power of power series, one 
should really understand both. Chapter 2 of this Handbook of Enumerative Combi- 
natorics is devoted to the analytic approach, which treats A(x) as an honest analytic 
function of x, and uses analytic properties of A(x) to derive combinatorial properties 
of a,. In this chapter we follow the algebraic approach, which treats A(x) as a for- 
mal algebraic expression, and manipulates it using the usual laws of algebra, without 
having to worry about any convergence issues. 


1.3.1 The ring of formal power series 


Enumerative combinatorics is full of intricate algebraic computations with power 
series, where justifying convergence is cumbersome, and usually unnecessary. In 
fact, many natural power series in combinatorics, such as )},597!x", only converge 
at 0, so analytic methods are not available to study them. For these reasons we often 
prefer to carry out our computations algebraically in terms of formal power series. 
We will see that even in this approach, analytic considerations are often useful. 

In this section we review the definition and basic properties of the ring of formal 
power series C|[x]]. For a more in-depth discussion, including the (mostly straight- 
forward) proofs of the statements we make here, see [149]. 


Formal power series. A formal power series is an expression of the form 
A(x) =ag tayx+anx?+---, ag,41,a2,... EC. 


Formally, this series is just a sequence of complex numbers dao,d1,d2,.... We will 
see that it is convenient to denote it A(x), but we do not consider it to be a function 
of x. 

Let C[[x]] be the ring of formal power series, where the sum and the product of 
A(x) i Yn>0 a,x" and B(x) = Yn>0 byx" are 


n 


A(x) + B(x) = (an +bn)x", A(x)B(x) = )° thnk Be 


n>0 n>0 \k= 
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It is implicit in the definition, but worth mentioning, that )°,>9 ax” = Lpsobnx” if 
and only if a, = by, for alln > 0. 
We define the degree of A(x) = Y,50 anx" to be the smallest n such that a, # 0. 
We also write 
[x"]A(x) := adn, A(0) := [x°JA(x) = ao. 


We also define formal power series inspired by series from analysis, such as 


f= PE, -tgt-y:=E", 40a Z Gn 


n>0°"" n>1 n>0 


for any complex number r, where (") :=r(r—1)-+-(r—n+1)/n!. 

The ring C[[x]] is commutative with O=0+0x+--- and1=1+0x+---.Itisan 
integral domain; that is, A(x)B(x) = 0 implies that A(x) = 0 or B(x) = 0. It is easy to 
describe the units: 


y a,x" is invertible = > aj #0. 
n>0 


For example, - = 1+x+27+--- because (1—x)(1+x+2x?+---) =140x+ 


1-x 
Ox? fos, 


Convergence. When working in C|[x]], we will not consider convergence of se- 
quences or series of complex numbers. In particular, we will never substitute a com- 
plex number x into a formal power series A(x). 

However, we do need a notion of convergence for sequences in C[|[x]]. We say 
that a sequence Ao(x),A1(x),A2(x),... of formal power series converges to A(x) = 
Ln>04nx" if limp. deg (An (x) — A(x)) = 0%; that is, if for any n €N, the coefficient 
of x" in A,,(x) equals a, for all sufficiently large m. This gives us a useful criterion 
for convergence of infinite sums and products in C[[]]: 


y A j(x) converges => lim degA ;(x) = 

j=0 aire 

1+A,(x)) converges => lim degA ;(x) = A;(0) =0 
[( j 8 arene er j 
j=0 ; 


For example, the infinite sum ),59(x + 1)"/2” does not converge in this topology. 
Notice that the coefficient of x° in this sum cannot be obtained through a finite com- 
putation; it would require interpreting the infinite sum ),,59 1/2”. On the other hand, 
the following infinite sum converges: 


y “( px) = 2 (1.6) 


! 
n>0"- m>1 m 


It is clear from the criterion above that this series converges; but why does it equal 
1-x? 
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Borrowing from analysis. In C|[x]], (1.6) is an algebraic identity which says that 
the coefficients of x in the left-hand side, for which we can give an ugly but finite 
formula, equal 1,—1,0,0,0,.... If we were to follow a purist algebraic attitude, we 
would give an algebraic or combinatorial proof of this identity. This is probably pos- 
sible, but intricate and rather dogmatic. A much simpler approach is to shift toward 
an analytic attitude, at least momentarily, and recognize that (1.6) is the Taylor series 


expansion of 
e7 leg(l—x) =1=% 


for |x| < 1. Then we can just invoke the following simple fact from analysis. 


Theorem 1.3.1 Jf two analytic functions are equal in an open neighborhood of 0, 
then their Taylor series at 0 are equal coefficient-by-coefficient; that is, they are equal 
as formal power series. 


Composition. The composition of two series A(x) = Y,sodnx" and B(x) = 
Yn>0 bnx” with bo = 0 is naturally defined to be 


A(B(x)) = dan (550) : 


Note that this sum converges if and only if bp = 0. Two very important special cases 
in combinatorics are the series and e8(*), 


1 
1—B(x) 
“Calculus.” We define the derivative of A(x) = ¥,50 nx" to be 


A'(x) = y (n+ L)dnyix". 


n>0 
This formal derivative satisfies the usual laws of derivatives, such as 
(A+B) =A'+B', (AB)' =A'B+AB’, [A(B(x))]’ =A‘ (B(x))B’ (x). 


We can still solve differential equations formally. For example, if we know that 
F'(x) = F(x) and F(O) = 1, then (logF(x))’ = F’(x)/F(x) = 1, which gives 
log F (x) =x and F(x) =e’. 

This concludes our discussion on the formal properties of power series. Now let 
us return to combinatorics. 


1.3.2 Ordinary generating functions 


Suppose we are interested in enumerating a family < = #UMUALU--- of com- 
binatorial structures, where .%, is a finite set consisting of the objects of “size” n. 
Denote by |a| the size of a € ©. The ordinary generating function of ./ is 


A(x) = y x!4l = ag +ayxtanx? +--- 
acd 


where a, is the number of elements of size n. 
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We are not interested in the philosophical question of determining what it means 
for & to be “combinatorial”; we are willing to call </ a combinatorial structure as 
long as a, is finite for all n. We consider two structures & and @ combinatorially 
equivalent, and write # = &, if A(x) = B(x). 

More generally, we may consider a family < where each element a is given 
a weight wt(a), often a constant multiple of x!@, or a monomial in one or more 
variables x1,...,x,. Again, we require that there are finitely many objects of any 
given weight. Then we define the weighted ordinary generating function of / to be 
the formal power series 


Awt(%1,---;4n) = y wt(a). 


ach 
Examples of combinatorial structures (with their respective size functions) are 
words on the alphabet {0,1} (length), domino tilings of rectangles of height 2 
(width), or Dyck paths (length). We may weight these objects by t* where k is, re- 
spectively, the number of 1s, the number of vertical tiles, or the number of returns to 
the x axis. 


1.3.2.1 Operations on combinatorial structures and their generating functions 


There are a few simple but very powerful operations on combinatorial structures, all 
of which have nice counterparts at the level of ordinary generating functions. Many 
combinatorial objects of interest may be built up from basic structures using these 
operations. 


Theorem 1.3.2 Let < and & be combinatorial structures. 
Ll. (€@ =A+4B: Disjoint union) If a @-structure of size n is obtained by choosing 
an of -structure of size n or a #-structure of size n, then 
C(x) =A(x) + B(x). 


This result also holds for weighted structures if the weight of a G-structure is 
the same as the weight of the respective &- or B-structure. 


2. (6 =A x B: Product) If a @-structure of size n is obtained by choosing an 
f-structure of size k and a &-structure of sizen—k for some k, then 
C(x) =A(x)B(x). 
This result also holds for weighted structures if the weight of a G-structure is 
the product of the weights of the respective &- and &-structures. 
3. (6 = Seq(F): Sequence) Assume |Bo| = 0. If a @-structure of size n is ob- 
tained by choosing a sequence of &-structures of total size n, then 
1 
C(x) = ———: 
() = Ea 


This result also holds for weighted structures if the weight of a G-structure is 
the product of the weights of the respective B-structures. 
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4. (€ = A 0: Composition) Compositional formula. Assume that |Bo| = 0. 
If a @-structure of size n is obtained by choosing a sequence of (say, k) B- 
structures of total size n and placing an &-structure of size k on this sequence 
of B-structures, then 

C(x) =A(B(x)). 


This result also holds for weighted structures if the weight of a @-structure is 
the product of the weights of the &-structure on its blocks and the weights of 
the B-structures on the individual blocks. 


5. (€ = I~: Inversion) Lagrange inversion formula. 


(a) Algebraic version. If A<~'* (x) is the compositional inverse of A(x) then 


“(Gat5). 


| =1, and let 


a 


nlx" |A<- > @) = [ 


(b) Combinatorial version. Assume || = 0, 


A(x) =x anx” — a3x° — asx" 
where ay is the number of &-structures of size n for n > 2. * 


Let an /-decorated plane rooted tree (or simply &-tree) be a rooted tree T 
where every internal vertex v has an ordered set D, of at least two “children,” 
and each set D, is given an f-structure. The size of T is the number of leaves. 


Let C(x) be the generating function for </-decorated plane rooted trees. Then 
CQ =AT Ge). 


This result also holds for weighted structures if the weight of a tree is the 
product of the weights of the &-structures at its vertices. 


Proof. — 1. is clear. The identity in 2. is equivalent to cy = Yy,axbn_—z~, which cor- 
responds to the given combinatorial description. Iterating 2., the generating func- 
tion for k-sequences of #-structures is B(x)*, so in 3. we have C(x) = Y,B(x)* = 
1/(1 —B(x)) and in 4. we have C(x) = Y;,a,B(x)*. The weighted statements follow 
similarly. 

5(b). Observe that, by the Compositional Formula, an .e7 o @ structure is either 
(i) an /-tree, or (ii) a sequence of k > 2 &-trees T,,...,7; with an /-structure on 
{T,...,T]}-. 

The structure in (ii) is equivalent to an .<-tree T, obtained by grafting 7),..., Tj, at 
a new root and placing the .-structure on its offspring (which contributes a negative 


“Here, to simplify matters, we introduced signs into A(x). Instead we could let A(x) be the ordinary 
generating function for .2-structures, but we would need to give each /-tree the sign (—1)’", where m is 
the number of internal vertices. Similarly, we could allow a; 4 1 at the cost of some factors of a; on the 
of -trees. 
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sign). This T also arises in (i) with a positive sign. These two appearances of T cancel 
each other out in A(C(x)), and the only surviving tree is the trivial tree e with one 
vertex, which only arises once and has weight x. 

5(a). Let a sprig be a rooted plane tree consisting of a path r= vj v2---y, = 1 
starting at the root r and ending at the leaf /, and at least one leaf hanging from each 
v; and to the right of v;,; for 1 <i<k—1. The trivial tree e is an allowable sprig 
with k = 1. 

An &-sprig is a sprig where the children of v; are given an -structure for 
1 <i<k-—1; its size is the number of leaves other than / minus 1. * The right panel 
of Figure 1.2 shows several .e/-sprigs. An .&@-sprig is equivalent to a sequence of 
-structures, with weights shifted by —1, so Theorem 1.3.2.3 tells us that 

1 1 1 


=--: = # of &-spri f si e 
AQ) x 1—(axta3e2+--) 2X! of of sprigs of size n)x 


Hence {x”~!](x/A(x))" = [x7 !](1/A(x))” is the number of sequences of n .o/-sprigs 
of total size —1 by Theorem 1.3.2.2. We need to show that 


n- (# of &-trees with n leaves) = (# of sequences of n o/-sprigs of total size —1) 


OO 


Figure 1.2 
The map from .2&/-trees to sequences of .e/-sprigs. 


SA kA 


An &/-tree T can be trimmed into a sequence of n -sprigs S1,...,Sn as follows. 
At each step, look at the leftmost leaf and the path P to its highest remaining ancestor. 
Remove P and all the branches hanging directly from P (which form an .-sprig), but 
do not remove any other vertices. Repeat this until the tree is completely decomposed 
into .&-sprigs. The total size of these sprigs is —1. Figure 1.2 shows a tree of weight 
got (1404 (-1)424(-)+(-D+(-)+(-)+(-1)+0+(-)) — yl, Notice that all the partial 
sums of the sum 5+ (—1) +0+(—1) +24 (-1) + (—1) +(-1) + (-1)4+ (-1) + 
0+ (—1) =~—1 are non-negative. 

Conversely, suppose we wish to recover the ./-tree corresponding to a sequence 
of sprigs S1,...,S, with |S;|-+---+ |S,| = —1. We must reverse the process, adding 


*We momentarily allow negative sizes, since the trivial ./-sprig ¢ has size —1. Thus we need to compute 
with Laurent series, which are power series with finitely many negative exponents. 
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S1,...,S, to T one at a time; at each step we must graft the new sprig at the leftmost 
free branch. Note that after grafting S),...,S, we are left with 1 + |S,|+---+|S,| 
free branches, so a sequence of sprigs corresponds to a tree if and only if the partial 
sums |S;|+--- + |S,| are non-negative for k = 1,...,1—1. Finally, it remains to 
observe that any sequence a),...,d, of integers adding to —1 has a unique cyclic 
shift aj,...,@y,41,---,a;-1 whose partial sums are all non-negative. Therefore, out of 
the n cyclic shifts of S,,...,S,, exactly one of them corresponds to an .e%/-tree. The 
desired result follows. a 


The last step of the proof above is a special case of the cycle lemma of Dvoret- 
sky and Motzkin [192, Lemma 5.3.7], which is worth stating explicitly. Suppose 
a1,.--,@y is a String of 1s and —1s with aj +---+a, =k > 0. Then there are exactly 
k cyclic shifts aj,aj41,...,4n,41,...,@i—1 whose partial sums are all non-negative. 


1.3.2.2 Examples 


Classical applications. With practice, these simple ideas give very easy solutions 
to many classical enumeration problems. 


1. (Trivial classes) It is useful to introduce the trivial class o having only one 
element of size 0, and the trivial class e having only one element of size 1. 
Their generating functions are | and x, respectively. 


2. (Sequences) The slightly less trivial class Seq = {0,0,e0,e0e0,...} = Seq(e) 
contains one set of each size. Its generating function is yx” = 1/(1—x). 


3. (Subsets and binomial coefficients) Let Subset consist of the pairs ([n],A) 
where 7 is a natural number and A is a subset of [n]. Let the size of that pair be 
n. A Subset-structure is equivalent to a word of length n in the alphabet {0, 1}, 
so Subset = Seq({0, 1}) where |0| = |1] = 1, and 


1 ALN 
Subset(x) = T-(@! +a!) = yi2 x. 


n>0 


We can use the extra variable y to keep track of the size of the subset A, by 
giving ({n],A) the weight x”y/4!. This corresponds to giving the letters 0 and 1 
weights x and xy respectively, so we get the generating function 


1 
Subsetwl®) = pay = E(t) 
n>k>0 


for the binomial coefficients (7) = ACI which count the k-subsets of [7]. 


From this generating function, we can easily obtain the main re- 
sults about binomial coefficients. Computing the coefficient of x”y* in 
(¥ ({)x"y*) (1 —x—xy) = 1 gives Pascal’s recurrence 


(C4). ee 
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with initial values (7) = (") = 1. Expanding Subsetw:(x) = jy = 


Yn>ox"(1+y)” gives the Binomial theorem 


aty"=¥ (Dye 


k=0 


4. (Multinomial coefficients) Let Words © Seq({1,...,k}) consist of the words 
in the alphabet {1,2,...,k}. The words of length n are in bijection with the 
ways of putting n numbered balls into k numbered boxes. The placements hav- 
ing a; balls in box i, where aj +----++ a, =n, are enumerated by the multino- 


co 2 ! 
mial coefficient ts ” *) = ica 


Giving the letter i weight x;, we obtain the generating function 


see 1 
Words* (x1,...,x%) = y eS 60) at capt = 


y,.--,aK20 


from which we obtain the recurrence 


n n—1 n—1 
Q,...,ak a, —1,d2,..., a Q1,---Ag_1,a,—1 


and the multinomial theorem 


5. (Compositions) A composition of n is a way of writing n =a, +--- +a 
as an ordered sum of positive integers a),...,az. For example, 523212 is a 
composition of 15. A composition is just a sequence of natural numbers, so 
Comp = Seq(N) where |a| = a. Therefore 


1 1-x 
_ ae _ gn-l ne 
1—(x+x2 4x3 4---)  1-2x 3 . 


n>1 


Comp(x) 


and there are 2”~! compositions of n. 
If we give a composition of n with k summands the weight x”y*, the weighted 
generating function is 
1 1-x n—-l1\ , 5 
TepaegS ma =), ees 
Lay ys) co beaaxy) ay ead 


Compyr(x) = 


so there are (eq) compositions of n with k summands. 


6. (Compositions into restricted parts) Given a subset A C N, an A-composition 
of nis a way of writing n as an ordered sum n = a, +-+-+ax where a),...,ay € 
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A. The corresponding combinatorial structure is A-Comp © Seq(A) where 
|a| =a, so 


1 
es (Daca x4) 
For example, the number of compositions of n into odd parts is the Fibonacci 
number F,,_;, because the corresponding generating function is 


A-Comp(x) = 


OddComp(x) : ecm 
xX). = —J = 
1-(xtx34+x54---)  L-x—x? = 


VM 
- 
% 


. (Multisubsets) Let Multiset’” be the collection of multisets consisting of possi- 


bly repeated elements of [m]. The size of a multiset is the number of elements, 
counted with repetition. For example, {1,2,2,2,3,5} is a multisubset of [7] 
of size 6. Then Multiset” = Seq({1}) x --- x Seq({m}), where |i] = 1 for 
i=1,...,m, so the corresponding generating function is 


Multiset” (x) = C= y=E (Tor 


and the number of multisubsets of [m] of size n is ((”)) := (—1)"(;"") = 


+n-1 7 
): 


. (Partitions) A partition of n is a way of writing n = a, +---+ a, as an 


unordered sum of positive integers a,,...,a,. We usually write the parts 
in weakly decreasing order. For example, 532221 is a partition of 15 into 
6 parts. Let Partition be the family of partitions weighted by xy where 
n is the sum of the parts and k is the number of parts. Then Partition = 
Seq({1}) x Seq({2}) x ---, where wt(i) = x'y for i= 1,2,..., so the corre- 
sponding generating function is 


V 1! | ; 
Partition(x,y) = (4) (5) (<5) ‘a 


There is no simple explicit formula for the number p(n) of partitions of n, 


although there is a very elegant and efficient recursive formula. Setting y= —1 

in the previous identity, and invoking Euler’s pentagonal theorem [2] 
[TlG-*) =14+ EC 1)/ (x! 3j-1)/2. 4 /(Bi41 v2 (1.7) 
n>0 j21 

we obtain 


p(n) = p(n—1) + p(n—2) — p(n—5) — p(n—7) + p(n— 12) + p(n— 15) 


where 1,2,5,7,12,15,22,26,... are the pentagonal numbers. 
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9. 


10. 


11. 


12. 


(Partitions into distinct parts) Let DistPartition be the family of partitions into 
distinct parts, weighted by xy where n is the sum of the parts and k is the 
number of parts. Then DistPartition © {1,1} x {2,2} x ---, where wt(i) = x'y 
and wt(i) = 1 for i= 1,2,..., so the corresponding generating function is 


DistPartition(x,y) = (1 +xy)(1 +27y)(1 +23y)-- 


(Partitions into restricted parts) It is clear how to adapt the previous generating 
functions to partitions where the parts are restricted. For example, the identity 
1 
Toy = it) +x7)(1+x4)(1+28)(1+219)--- 
—x 


expresses the fact that every positive integer can be written uniquely in binary 
notation, as a sum of distinct powers of 2. The identity 
1 1 1 1 


1 14:2x2)(1423)(1 4x4)... = . ‘ : aa 
(Ita) +a (+x) +2") l—-x 1-8 1-x 1- x’ , 


which may be proved by writing 1 +x* = (1 —x*)/(1 —*), expresses that the 
number of partitions of into distinct parts equals the number of partitions of 
n into odd parts. 


(Partitions with restrictions on the size and the number of parts) Let p<,(n) 
be the number of partitions of n into at most k parts. This is also the number 
of partitions of n into parts of size at most k. To see this, represent a partition 
n=a,+---+a; as a left-justified array of squares, where the ith row has a; 
squares. Each partition A has a conjugate partition A’ obtained by exchanging 
the rows and the columns of the Ferrers diagram. Figure 1.3 shows the Ferrers 
diagram of 431 and its conjugate partition 3221. It is clear that A has at most k 
parts if and only if A’ has parts of size at most k. 


From the previous discussion it is clear that 


1 1 1 
Y p<i(n)x" = 7 


; sot 7c: 
0 x |l-x 1—-x 


Now let p<j,.<x(n) be the number of partitions of n into at most j parts of size 
at most k. Then 


nies (1=x)(1—27)-s- (1 —x/**) 
Y Psick(n)x" = (—)0—2):+0—#)-0—x)0—x2) = (a). 


n>0 


This is easily proved by induction, using that p<j.<¢(n) = p<j.<k—1(n) + 
P<jck(n—k). 

(Even and odd partitions) Setting y = —1 into the generating function for par- 
titions into distinct parts of Example 9, we get 


Y" (edp(2) — odp())x” = [] (1 —2/) = 1 Can Sag 


n>0 j21 
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Figure 1.3 
The Ferrers diagrams of the conjugate partitions 431 and 3221. 


13. 


where edp(n) (respectively, odp(7)) counts the partitions of n into an even 
(respectively, odd) number of distinct parts. Euler’s pentagonal formula (1.7) 
says that edp(n) — odp(n) equals 0 for all n except for the pentagonal numbers, 
for which it equals | or —1. 


There are similar results for partitions into distinct parts coming from a given 
set S. 


e When S is the set of Fibonacci numbers, the coefficients of the generating 
function 


[Ja-x*)=1 x ext ty7 x8 4x1! x)2 xB 4yl44.. 
n>1 


are also equal to 0,1, or —1. [5, 168] 


e This is also true for any “k-Fibonacci sequence” S = {a1,az,...} given 
by dy = Qn—1 +++++y-, forn >k and aj > aj-j+---+a; forl<j<k. 
[67] 


e The result also holds trivially for S = {2/ : j € N} since there is a unique 
partition of any n into distinct powers of 2. 


These three results seem qualitatively different from (and increasingly less 
surprising than) Euler’s result, as these sequences S grow much faster than 
{1,2,3,...}, and S-partitions are sparser. Can more be said about the sets S of 
positive integers for which the coefficients of [],¢5(1 — x") are all 1,0 or —1? 


(Set partitions) A set partition of a set S is an unordered collection of pair- 
wise disjoint sets S,,...,5; whose union is S. The family of set partitions 
with k parts is SetPartition’ & e x Seq({1}) x e x Seq({1,2}) x ---xex 
Seq({1,2,...,k}), where the singleton e and all numbers i have size 1. To 
see this, we regard a word such as w = e 11 e 1221 e31 as an instruction man- 
ual to build a set partition $1,...,5,. The jth symbol w; tells us where to put 
the number j: if w; is a number h, we add j to the part S;,; if w; is the ith e, 
then we add j to a new part S;. The sample word above leads to the partition 
{1,2,3,5,8, 11}, {4,6,7}, {9, 10}. This process is easily reversible. It follows 
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14. 


that 
x x x 


S(n.k)x" = yy Sead 
YL S@.ke" = 7 Toa, (=e 


n>0 


where S(n,k) is the number of set partitions of [n] into k parts. These numbers 
are called the Stirling numbers of the second kind. 

The equation (1 — kx) ¥,59S(n,k)x" = xLnsoS(n,k — 1)x” gives the recur- 
rence 


S(n,k) =kS(n—1,k) +S(n—-1,k-1), 1 Sk <a, 


with initial values S(n,0) = S(n,n) = 1. Note the great similarity with Pascal’s 
recurrence. 


(Catalan structures) It is often said that if you encounter a new family of math- 
ematical objects, and you have to guess how many objects of size n there are, 
you should guess “the Catalan number C,, = a () .’ The Catalan family has 
more than 200 incarnations in combinatorics and other fields [185, 192]; let us 
see three important ones. 


(a) (Plane binary trees) A plane binary tree is a rooted tree where every 
internal vertex has a left and a right child. Let PBTree be the family of 
plane binary trees, where a tree with n internal vertices (and necessarily 
n-+1 leaves) has size n. A tree is either the trivial tree o of size 0, or the 
grafting of a left subtree and a right subtree at the root e, so PBTree = 
o+(PBTree x e x PBTree). It follows that the generating function for 
plane binary trees satisfies 


T(x) =14+T7(x)xT(x). 
We may use the quadratic formula * and the binomial theorem to get 


1—-V1—-4x (*") z 
x", 
n>0 n 


1 
On ke 


It follows that the number of plane binary trees with n internal vertices 

(and n+ 1 leaves) is the Catalan number C,, = mati = . 

(b) (Triangulations) A triangulation of a convex polygon is a subdivision 
into triangles using only the diagonals of P. A triangulation of an (n +-2)- 
gon has n triangles; we say it has size n. If we fix an edge e of P, then a 
triangulation of P is obtained by choosing the triangle T that will cover 
e, and then choosing a triangulation of the two polygons to the left and 
to the right of T. Therefore Triang ~ o+(Triang x e x Triang) and the 
number of triangulations of an (n+ 2)-gon is also the Catalan number 
Ch. 


*Since this is the first time we are using the quadratic formula, let us do it carefully. Rewrite the equation 
as (1 — 2x7 (x))? = 1—4x , or (1 — 2x7 (x) — V1 — 4x) (1 — 2x7 (x) + V1 — 4x) = 0. Since C[[x]] is an 
integral domain, one of the factors must be 0. From the constant coefficients we see that it must be the 
first factor. 
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(c) (Dyck paths) A Dyck path P of length n is a path from (0,0) to (2,0) 
that uses the steps (1,1) and (1,—1) and never goes below the x-axis. Say 
P is irreducible if it touches the x-axis exactly twice, at the beginning 
and at the end. Let D(x) and I(x) be the generating functions for Dyck 
paths and irreducible Dyck paths. 
A Dyck path is equivalent to a sequence of irreducible Dyck paths. Also, 
an irreducible path of length n is the same as a Dyck path of length n— 1 
with an additional initial and final step. Therefore 


from which it follows that D(x) = /-4* vias as well, and the number of 


Dyck paths of length n is also the Catalan number. 


Generatingfunctionology gives us fairly easy algebraic proofs that these three 
families are enumerated by the Catalan numbers. Once we have discovered 
this fact, the temptation to search for nice bijections is hard to resist. 


Our algebraic analysis suggests a bijection @ from (b) to (a). The families of 
plane binary trees and triangulations grow under the same recursive recipe, and 
so we can let the bijection grow with them, mapping a triangulation T x e x T’ 
to the tree @(T) x e x @(T’). A non-recursive description of the bijection is 
the following. Consider a triangulation T of the polygon P, and fix an edge e. 
Put a vertex inside each triangle of T, and a vertex outside P next to each edge 
other than e. Then connect each pair of vertices separated by an edge. Finally, 
root the resulting tree at the vertex adjacent to e. This bijection is illustrated in 
Figure 1.4. 


Figure 1.4 
The bijection from triangulations to plane binary trees. 


Algebraic and Geometric Methods in Enumerative Combinatorics 23 


= EIN 


Figure 1.5 
The bijection from plane binary trees to Dyck paths. 


15. 


A bijection from (a) to (c) is less obvious from our algebraic computations, but 
is still not difficult to obtain. Given a plane binary tree T of size n, prune all 
the leaves to get a tree J’ with n vertices. Now walk around the periphery of 
the tree, starting on the left side from the root, and continuing until we traverse 
the whole tree. Record the walk in a Dyck path D(T) : every time we walk up 
(respectively, down) a branch we take a step up (respectively, down) in D(T). 
One easily checks that this is a bijection. See Figure 1.5 for an illustration. 


Even if it may be familiar, it is striking that two different (and straightforward) 
algebraic computations show us that two families of objects that look quite 
different are in fact equivalent combinatorially. Although a simple, elegant bi- 
jection can often explain the connection between two families more transpar- 
ently, the algebraic approach is sometimes simpler, and better at discovering 
such connections. 


(k-Catalan structures) Let PTree, be the class of plane k-ary trees, where 
every vertex that is not a leaf has k ordered children; let the size of such a tree 
be its number of leaves. In the sense of Theorem 1.3.2.5(a), this is precisely 
an ./-tree, where of = {e, e“} consisting of one structure of size 1 and one of 
size k. Therefore PTree, = (x —x*)<—!>. Lagrange inversion then gives 


m[x™A!? (x) = [x (5) = ey aa ee 


n 


It follows that a plane k-ary tree must have m = (k — 1)n+ 1 leaves for some 
integer n, and the number of such trees is the k-Catalan number 


1 kn 
k 
d= gaa (*). 


This is an alternative way to compute the ordinary Catalan numbers C,, = (See 


The k-Catalan number C* also has many different interpretations [100]; we 
mention two more. It counts the subdivisions of an (n(k — 1) +2)-gon P into 
(necessarily n) (k+ 1)-gons using diagonals of P, and the paths from (0,0) 
to (n,(k —1)n) with steps (0,1) and (1,0) that never rise above the line y = 
(k—1)x. 
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Other applications. Let us now discuss a few other interesting applications that 
illustrate the power of Theorem 1.3.2. 


16. (Motzkin paths) The Motzkin number ™,, is the number of paths from (0,0) 
to (n,0) using the steps (1,1), (1,—1), and (1,0), which never go below the 
x-axis. Imitating our argument for Dyck paths, we obtain a formula for the 
generating function 


1 _ 1-x-V1-2x—3x2 


NO—TSGaaiG@a 7 Te me 


The quadratic equation x*M* + (x — 1)M +1 =0 gives rise to the quadratic re- 
currence M, = M,_1 +Y.;M;M,_2-;. The fact that M(x) satisfies a polynomial 
equation leads to a more efficient recurrence 


(n+2)My = (2n+ 1)My-1 + (3n—3)My_2. 
We will see this in Section 1.3.4.2. 


17. (Schréder paths) The (large) Schréder number r,, is the number of paths from 
(0,0) to (2n,0) using steps NE = (1,1),SE = (1,—1), and E = (2,0) which 
stays above the x-axis. Their generating function satisfies R(x) = 1/(1 —x— 
xR(x)), and therefore 


l—x-—V1—6x+x2 
R(x) = ae : 


Theorem 1.3.2.3 is useful when we are counting combinatorial objects that “fac- 
tor” uniquely into an ordered “product” of “irreducible” objects. It tells us that we 
can count all objects if and only if we can count the irreducible ones. We have already 
used that idea several times; let us see it in action in some other examples. 


18. (Domino tilings of rectangles) In Section 1.2 we let a, be the number of 
domino tilings of a 2 x n rectangle. Such a tiling is uniquely a sequence of 
blocks, where each block is either a vertical domino (of width 1) or two hori- 
zontal dominoes (of width 2). This truly explains the formula: 


1 


AG) = Tara) 


Similarly, if dm is the number of domino tilings of a 2 x n rectangle using v 
vertical tiles, we immediately obtain 


1 


OV xe = 
ey ae 1— (vx +x?) 


Sometimes the enumeration of irreducible structures is not immediate, but still 
tractable. 
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19. (Monomer-dimer tilings of rectangles) Let T(2,n) be the number of tilings 
of a 2 xn rectangle with dominoes and unit squares. Say a tiling is irre- 
ducible if it does not contain an internal vertical line from top to bottom. Then 
Tilings = Seq(IrredTilings). It now takes some thought to recognize the irre- 
ducible tilings in Figure 1.6. 


i A es er eae ee ee | 


Figure 1.6 
The irreducible tilings of 2 x n rectangles into dominoes and unit squares. 


There are three irreducible tilings of length 2, and two of every other length 
greater than or equal to |. Therefore 


LTO nr = : = en 
ao ae 1 (2x+3x2 +203 42x44---)  1-3x-2? +33" 


We will see in Theorem 1.3.5.2 that this gives T(2,n) ~ c- a” where a & 
3.214... is the inverse of the smallest positive root of the denominator. 


Sometimes the enumeration of all objects is easier than the enumeration of the irre- 
ducible ones. In that case we can use Theorem 1.3.2.3 in the opposite direction. 


20. (Irreducible permutations) A permutation of |n] is irreducible if it does not 
factor as a permutation of {1,...,m} and a permutation of {m+1,...,n} for 
1<m <n; that is, if 2([m]) F [m] for all 1 <m <n. Clearly every permutation 
factors uniquely into irreducibles, so 


1 
1x? = ——________, 
py 1 — IrredPerm(x) 


n>0 
This gives the series for IrredPerm. 


There are many interesting situations where it is possible, but not at all trivial, to 
decompose the objects that interest us into simpler structures. To a combinatorialist 
this is good news, the techniques of this section are useful tools, but are not enough. 
There is no shortage of interesting work to do. Here is a great example. 


21. (Domino towers) [91, 31, 221] A domino tower is a stack of horizontal 2 x 1 
bricks in a brickwork pattern, so that no brick is directly above another brick, 
such that the bricks on the bottom level are contiguous, and every higher brick 
is (half) supported on at least one brick in the row below it. Let the size of a 
domino tower be the number of bricks. See Figure 1.7 for an illustration. 
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Figure 1.7 
A domino tower of 19 bricks. 


Remarkably, there are 3”~! domino towers consisting of n bricks. Equally re- 
markably, no simple bijection is known. The nicest argument known is as fol- 
lows. 


| Ee) 
[=] 


EJ 


Figure 1.8 
The decomposition of a domino tower into a pyramid and three half-pyramids. 


We decompose a domino tower x into smaller pieces, as illustrated in Fig- 
ure 1.8. Each new piece is obtained by pushing up the leftmost remaining 
brick in the bottom row, dragging with it all the bricks encountered along the 
way. The first piece p will be a pyramid, which we define to be a domino 
tower with only one brick in the bottom row. All subsequent pieces hy, ... , hg 
are half-pyramids, which are pyramids containing no bricks to the left of 
the bottom brick. This decomposition is reversible. To recover x, we drop 
hy, hy_1,--.,h1, p from the top in that order; each piece is dropped in its cor- 
rect horizontal position, and some of its bricks may get stuck on the previ- 
ous pieces. This shows that the corresponding combinatorial classes satisfy 
X = Px Seq(H). 

Similarly, we may decompose a pyramid p into half-pyramids, as shown 
in Figure 1.9. Each new half-pyramid is obtained by pushing up the left- 
most remaining brick (which is not necessarily in the bottom row), drag- 


ging with it all the bricks that it encounters along the way. This shows that 
P= Seqs)(H):=H+(HxH)+---. 
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Figure 1.9 
A pyramid and its decomposition into half-pyramids. 


Finally consider a half-pyramid h; there are two cases. If there are other bricks 
on the same horizontal position as the bottom brick, consider the lowest such 
brick, and push it up, dragging with it all the bricks it encounters along the way, 
obtaining a half-pyramid h,. Now remove the bottom brick; what remains is 
a half-pyramid hz. This is shown in Figure 1.10. As before, we can recover 
h from hy, and hz. On the other hand, if there are no bricks above the bottom 
brick, removing the bottom brick leaves either a half-pyramid or the empty set. 
Therefore H =~ (H xex H)+(exH)+e. 


| 
= | 
Figure 1.10 


A (non-half-pyramid) pyramid and its decomposition into two half-pyramids and a 
bottom brick. 


The above relations correspond to the following identities for the correspond- 
ing generating functions: 


ee poste H =xH?+xH+ 
Tee ee Tee yg ee 


xX 
Surprisingly cleanly, we obtain X (x) =x/(1 —3x) =Yn513” !x". This proves 
that there are 3”~! domino towers of size n. 

Although we do not need this here, it is worth noting that half-pyramids are 
enumerated by Motzkin numbers; their generating functions are related by 
H (x) =xM(x). 
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1.3.3 Exponential generating functions 


Ordinary generating functions are usually not well suited for counting combinatorial 
objects with a labeled ground set. In such situations, exponential generating functions 
are a more effective tool. 

Consider a family WY = mUMUA2U--- of labeled combinatorial struc- 
tures, where .%, consists of the structures that we can place on the ground set 
[n] = {1,...,} (or, equivalently, on any other labeled ground set of size n). If ae &H, 
we let |a| = 7 be the size of a. We also let a, be the number of elements of size n. 
The exponential generating function of © is 


xlal x9 x! x a 
been. 


A= Mog Tela so a 


acd lal! 


We may again assign a weight wt(a) to each object a, usually a monomial in 
variables x1,...,%, and consider the weighted exponential generating function of 
&f to be the formal power series 


dal 
Awt(*1, pa Xn X) 3 wt(a)—. 


Examples of combinatorial structures (with their respective size functions) are per- 
mutations (number of elements), graphs (number of vertices), or set partitions (size 
of the set). We may weight these objects by t* where k is, respectively, the number 
or cycles, the number of edges, or the number of parts. 


1.3.3.1 Operations on labeled structures and their exponential generating 
functions 


Again, there are some simple operations on /abeled combinatorial structures, which 
correspond to simple algebraic operations on the exponential generating functions. 
Starting with a few simple structures, these operations are sufficient to generate many 
interesting combinatorial structures. This will allow us to compute the exponential 
generating functions for those structures. 


Theorem 1.3.3 Let & and & be labeled combinatorial structures. 


1. (€@ =A4+8: Disjoint union) If a @-structure on a finite set S is obtained by 
choosing an <&-structure on S or a &-structure on S, then 


C(x) =A(x) + B(x). 


This result also holds for weighted structures if the weight of a G-structure is 
the same as the weight of the respective &- or B-structure. 


2. (6 = A * B: Labeled Product) If a @-structure on a finite set S is obtained 
by partitioning S into disjoint sets S, and Sz and putting an f-structure on S, 
and a &-structure on So, then 
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This result also holds for weighted structures if the weight of a G-structure is 
the product of the weights of the respective &- and &-structures. 


3. (© = Seq, (&): Labeled Sequence) Ifa @-structure on a finite set S is obtained 
by choosing an ordered partition of S into a sequence of blocks and putting a 
&-structure on each block, then 


1 
C(x) = ——-—.. 
©) = TE 
This result also holds for weighted structures if the weight of a G-structure is 
the product of the weights of the respective B-structures. 


4. (© = Set(B): Set) Exponential formula. If a @-structure on a finite set S 
is obtained by choosing an unordered partition of S into a set of blocks and 
putting a B-structure on each block, then 


C(x) = 80), 


This result also holds for weighted structures if the weight of a G-structure is 
the product of the weights of the respective B-structures. 


In particular, if cy (n) is the number of @-structures of an n-set that decompose 
into k components (&-structures), we have 


n 


Xx 
Xcel) Gy = oF) = Clay? 


n,k,>0 


5. (€ = & 0 B: Composition) Compositional formula. If a @-structure on a 
finite set S is obtained by choosing an unordered partition of S into a set of 
blocks, putting a &-structure on each block, and putting an f-structure on 
the set of blocks, then 

C(x) =A(BQ)). 
This result also holds for weighted structures if the weight of a @-structure 
is the product of the weights of the <-structure on its set of blocks and the 
weights of the B-structures on the individual blocks. 


Again, Theorem 1.3.4.4 is useful when we are counting combinatorial objects 
that “decompose” uniquely as a set of “indecomposable” objects. It tells us that we 
can count all objects if and only if we can count the indecomposable ones. Amaz- 
ingly, we also obtain for free the finer enumeration of the objects by their number of 
components. 


Proof. 1. is clear. The identity in 2. is equivalent to c, = Y, (1) aPn—ke which 
corresponds to the given combinatorial description. Iterating 2., we see that the ex- 
ponential generating functions for k-sequences of 4-structures is B(x)*, and hence 
the one for k-sets of A-structures is B(x)*/k!. This readily implies 3, 4, and 5. The 
weighted statements follow similarly. a 


The following statements are perhaps less fundamental, but also useful. 
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Theorem 1.3.4 Let & be a labeled combinatorial structure. 


1. (€ =&,: Shifting) If a @-structure on S is obtained by adding a new element 
t to S and choosing an f-structure on SU {t}, then 


C(x) =A'(x). 


2. (6 = &,: Rooting) If a @-structure on S is a rooted .-structure, obtained by 
choosing an f-structure on S and an element of S called the root, then 


C(x) = xA(x). 


3. (Sieving by parity of size) If the @-structures are precisely the &-structures of 
even size, 


A A(- 
(x) — AWA), 
2 
4. (Sieving by parity of components) Suppose <-structures decompose uniquely 
into components, so & = Set(#) for some ZB. If the @-structures are the &- 
structures having only components of even size, 


C(x) = VA(@)A(—2). 


5. (Sieving by parity of number of components) Suppose &f -structures decompose 
uniquely into components, so & = Set(©) for some ©. If the @-structures are 
precisely the /-structures having an even number of components, 


Similar sieving formulas hold modulo k for any k € N. 


Proof. We have cy = an41 in 1., Cp = Ndy in 2., and cy = 5(an + (-1)"a,) in 3., 
from which the generating function formulas follow. Combining 3. with the Expo- 
nential Formula we obtain 4. and 5. 

Similarly we see that the generating function for </-structures whose size is a 
multiple of k is i (A(x) +A(@x) +---+A(@*!x)) where @ is a primitive kth root of 
unity. If we wish to count elements of size i mod k, we use |. to shift this generating 
function i times. a 


1.3.3.2 Examples 


Classical applications. Once again, these simple ideas give very easy solutions to 
many classical enumeration problems. 
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1. (Trivial classes) Again we consider the trivial classes o with only one element 
of size 0, and e with only one element of size 1. Their exponential generating 
functions are | and x, respectively. 


2. (Sets) A slightly less trivial class of Set contains one set of each size. We also 
let Sets; denote the class of non-empty sets, with generating function e* — 1. 
The exponential generating functions are 


Set(x) = e*, Sets: (x) =e*—1. 


3. (Set partitions) In Section 1.3.2.2 we found the ordinary generating function 
for Stirling numbers S(n,k) for a given k; but in fact it is easier to use ex- 
ponential generating functions. Simply notice that SetPartition = Set(Set>1), 
and the Weighted Exponential Formula then a 


SetPartition(x, y) = F.- SI S(n, k)~ k_ py(e— 1). 
n,k>0 


4. (Permutations) Let Perm, consist of the n! permutations of [n]. A permutation 
is a labeled sequence of singletons, so Perm = Seq,.(e), and the generating 
function for permutations is 


x” 
Perm(x) = ) n!— = 


5. (Cycles) Let Cycle, consist of the cyclic orders of |n]. These are the ways of 
arranging 1,...,” around a circle, where two orders are the same if they differ 
by a rotation of the circle. There is an n-to-1 mapping from permutations to 
cyclic orders obtained by wrapping a permutation around a circle, so 


Cycle(x) = Vi(n- 1)!x"/n! = —log(1—x). 


n 


There is a more indirect argument that will be useful to us later. Recall that a 
permutation z can be written uniquely as a (Commutative) product of disjoint 
cycles of the form (i, (i), 27(i),...,2*"!(i)), where k is the smallest index 
such that *(i) = i. For instance, the permutation 835629741 can be written in 


cycle notation as (18469)(235)(7). Then Perm = Set(Cycle) so 1/(1 —x) = 
eLycle(x) 


6. (Permutations by number of cycles) The (signless) Stirling number of the 
first kind c(n,k) is the number of permutations of n having k cycles. The 
Weighted Exponential Formula gives 


Y cab) “yt = erentes (TL) = Y y+): (ytn—1)> 


n,k>0 n>0 


xn 
nt 


so the Stirling numbers c(,k) of the first kind are the coefficients of the poly- 
nomial y(y+1)---(y+n-—1). 
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Other applications. The applications of these techniques are countless; let us con- 
sider a few more applications, old and recent. 


the 


10. 


(Permutations by cycle type) The type of a permutation 7 € S, is type(w) = 
(c1,---,Cn) where c; is the number of cycles of length i. For indeterminates t = 
(t1,.--,tn), ict tpe() — 7°! ...¢°, The cycle indicator of the symmetric group 
Sn is Zp = a ae s, thypetw () The Weighted Exponential Formula immediately 
gives 

y Zyx" = giitthx 2+? [3+ 

n>0 


Let us discuss two special cases of interest. 


(Derangements) A derangement of [n] is a permutation such that 2(i) 4 i 
for all i € [n]. Equivalently, a derangement is a permutation with no cycles of 
length 1. It follows that Derangement = Set(Cycle,,), so the number d,, of 
derangements of [n] is given . 7 


Derangement(x =P a", ig 8) me ge? fe eo, 
n>0 


which leads to the explicit formula 


‘ees ae | 1 n! 
—n! | be Sask ~ 
a,=n'(1 (i). oe at +5) e- 


(Involutions) An involution of [n] is a permutation w such that w7 is the iden- 
tity. Equivalently, an involution is a permutation with cycles of length 1 and 2, 
so the number i, of involutions of [n] is given by 


Note that Inv’ (x) = («+ 1)Inv(x), which gives in = in_1 + (n— 1)in—2. In Sec- 
tion 1.3.4.2 we will explain the theory of D-finite power series, which turns 
differential equations for power series into recurrences for the corresponding 
sequences. 


(Trees) A tree is a connected graph with no cycles. Consider a “birooted” 
tree (T,a,b) on [n] with two (possibly equal) root vertices a and b. Regard the 
unique path a = vg, V1,..., Vk = basa “spine” for T; the rest of the tree consists 
of rooted trees hanging from the v;s; direct their edges toward the spine. Now 
regard v,...vz aS a permutation in one-line notation, and rewrite it in cycle 
notation, while continuing to hang the rooted trees from the respective v;’s. 
This transforms (T,a,b) into a directed graph consisting of a disjoint collection 
of cycles with trees directed toward them. Every vertex has outdegree 1, so this 
defines a function f : [n] > |[n]. A moment’s thought will convince us that this 
is a bijection. Therefore there are n” birooted trees on [n], and hence there are 
n"~? trees on |n]. See Figure 1.11 for an illustration. 
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Figure 1.11 
A tree on [8] birooted at a = 3 and b =S, and the corresponding function f : [8] — [8]. 


11. (Trees, revisited) Let us count trees in a different way. Let a rooted tree be 
a tree with a special vertex called the root, and a planted forest be a graph 
with no cycles where each connected component has a root. Let t,,7, f, and 
T (x), R(x), F(x) be the sequences and exponential generating functions enu- 
merating trees, rooted trees, and planted forests, respectively. 


Figure 1.12 
A rooted tree seen as a root attached to the roots of a planted forest. 


Planted forests are vertex-disjoint unions of rooted trees, so F (x) = eR), Also, 
as illustrated in Figure 1.12, a rooted tree T consists of a root attached to the 
roots of a planted forest, so R(x) = xF (x). It follows that x = R(x)e~®, so 


R(x) = (xe™*)<- >. 


In 


Lagrange Inversion gives n- nb 


pan fr= (n+ 1)", sae 


We state a finer enumeration; see [192, Theorem 5.3.4] for a proof. The de- 
gree sequence of a rooted forest on [n] is (deg 1,...,degm) where degi is the 
number of children of i. For example the degree sequence of the rooted tree in 
Figure 1.12 is (3,1,0,0,0,0,2,0,2). Then the number of planted forests with 
a given degree sequence (d),...,d,) and (necessarily) k = n—(d, +---+d,) 
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n—1 n—k 

k—-1/) \d,...,;dy] 
The number of forests on |] is given by a more complicated alternating sum; 
see [197]. 


components is 


(Permutations, revisited) Here is an unnecessarily complicated way of prov- 
ing there are n! permutations of [n]. A permutation 2 of [n + 1] decomposes 
uniquely as a concatenation 7 = L(n+1)R for permutations L and R of two 
complementary subsets of [n]. Therefore Shift(Perm) = (Perm) x (Perm), and 
the generating function P(x) for permutations satisfies P/(x) = P(x)? with 
P(0) = 1. Solving this differential equation gives P(x) = ce = Lnoon!. 


(Alternating permutations) The previous argument was gratuitous for permu- 
tations, but it will now help us to enumerate the class Alt of alternating per- 
mutations w, which satisfy w; < w2 > w3 < w4 >---. The Euler numbers are 
E, = |Alt,|; let E(x) be their exponential generating function. We will need 
the class RevAlt of permutations w with w, > w2 < w3 > w4 <-:--. The map 
W=W1...WnOw = (n+1—wy1)...(2+1—w,) on permutations of [n] shows 
that Alt = RevAlt. 


Now consider alternating permutations L and R of two complementary subsets 
of [n]. For n > 1, exactly one of the permutations L(n + 1)R and L’(n + 1)R is 
alternating or reverse alternating, and every such permutation arises uniquely 
in that way. For n = 0 both are alternating. Therefore Shift(Alt + RevAlt) = 
Alt * Alt +0, so 2E’(x) = E(x)? +1 with E(0) = 1. Solving this differential 


equation we get 
x” 
E(x)=) E,— 


n>0 


a = secx + tanx. 
Therefore secx and tanx enumerate the alternating permutations of even and 
odd length, respectively. The Euler numbers are also called secant and tangent 
numbers. This surprising connection allows us to give combinatorial interpre- 
tations of various trigonometric identities, such as 1 + tan? x = sec? x. 

(Graphs) Let g(v) and gconn(v) be the number of simple graphs and connected 
graphs on [v], respectively. (A graph is simple if it contains no loops and no 
multiple edges.) The Exponential Formula tells us that their exponential gener- 
ating functions are related by G(x) = eGo (*), In this case it is hard to count the 
connected graphs directly, but it is easy to count all graphs: to choose a graph 


we just have to decide whether each edge is present or not, so g(v) = 2(2). This 


gives us 
all 
y 200%) 
v! 


v>0 


x’ 
» Sconn(V) = log ( 


v>0 


We may easily adjust this computation to account for edges and components. 
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15. 


There are Coe) graphs on [v] with e edges; say g(v,c,e) of them have c 
components, and give them weight y°z°. Then 


Vv y y 
y av, cve) ye = ( y Gra = F(x,1+z) 


v,c,e>0 v,e>0 


where 


F(a,B) =)" 


n>0 


is the deformed exponential function of [179]. 


(Signed graphs) A signed graph G is a set of vertices, with at most one “pos- 
itive” edge and one “negative” edge connecting each pair of vertices. We say 
G is connected if and only if its underlying graph G (ignoring signs) is con- 
nected. A cycle in G corresponds to a cycle of G; we call it balanced if it 
contains an even number of negative edges, and unbalanced otherwise. We 
say that G is balanced if all its cycles are balanced. Let s(v,c+,c_,e) be the 
number of signed graphs with v vertices, e edges, c, balanced components, 
and c_ unbalanced components; we will need the generating function 
CL C~ Ce 


x” 
Sy 2= Yo svycr,c_,e) yf yz 
G signed graph Ve 


in order to carry out a computation in Section 1.8.9; we follow [11]. 


Let S(x,y+,y_,z), B(x, y+,z), C+(x,z), and C_(x,z) be the generating func- 
tions for signed, balanced, connected balanced, and connected unbalanced 
graphs, respectively. The Weighted Exponential Formula gives 


— pt. — py¢Cyt+y—C_ 
B=e+ t, S=ertrTy ‘ 


so if we can compute C, and C_ we will obtain B and S. In turn, these equa- 
tions give 


1 
C4(x,2) = 5 log B(x, 2,2), C+(x,2) +C-(252) = log S(x, 1, L,2)3 


and we now compute the right-hand side of these two equations. (In the first 
equation, we set t, = 2 because, surprisingly, B(x,2,z) is easier to compute 
than B(x, 1,z).) One is easy: 


Steytst2)= (PY) = Fes +a) 


e,v>0 e 


For the other one, we count balanced signed graphs by relating them with 
marked graphs, which are simple graphs with a sign + or — on each ver- 
tex. [95] A marked graph M gives rise to a balanced signed graph G by as- 
signing to each edge the product of its vertex labels. Furthermore, if G has c 
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Figure 1.13 
The two marked graphs that give rise to one balanced signed graph. 


components, then it arises from precisely 2° different marked graphs, obtained 
from M by choosing some connected components and changing their signs. 
This correspondence is illustrated in Figure 1.13. It follows that B(x,2y,z) = 
YB balanced 2° D(v, ¢, e) xyz? = Yu markea (1, €,€) x yz? is the generating func- 
tion for marked graphs, and hence B(x,2,z) may be computed easily: 


v Vv 
B22) =, i) see F(2x,1+z). 
ar we v! 
Putting these equations together yields 


S(x,¥4,¥—52) = F(2x,1+2)0+ OP F(x, (1 +z). 


1.3.4 Nice families of generating functions 


In this section we discuss three nice properties that a generating function can have: 
being rational, algebraic, or D-finite. Each one of these properties gives rise to useful 
properties for the corresponding sequence of coefficients. 


1.3.4.1 Rational generating functions 


Many sequences in combinatorics and other fields satisfy three equivalent proper- 
ties: They satisfy a recursive formula with constant coefficients, they are given by an 
explicit formula in terms of polynomials and exponentials, and their generating func- 
tions are rational. We understand these sequences very well. The following theorem 
tells us how to translate any one of these formulas into the others. 


Theorem 1.3.5 [194, Theorem 4.1.1] Let ao,a,,a2,... be a sequence of complex 
numbers and let A(x) = Yy»s0Gnx" be its ordinary generating function. Let q(x) = 
L+eyx+-+-+egxt = (1 —ryx)" --- (1 — rex)“ be a complex polynomial of degree 
d. The following are equivalent: 


1. The sequence satisfies the linear recurrence with constant coefficients 


An +c1dn_1 +++: +Cgan_-gd =0 (n> d). 
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2. There exist polynomials f,(x),...,f;(x) with deg fi(x) < dj for 1 <i<n such 
that 
Gn = filn)ri te + fe(n) ry. 


3. There exists a polynomial p(x) with deg p(x) < d such that A(x) = p(x)/q(x). 


Notice that Theorem 1.3.5.2 gives us the asymptotic growth of a, immediately. 
Let us provide more explicit recipes. 


(1 = 2) Extract the inverses 7; of the roots of g(x) = 1+c)x+---+cgx and their 
multiplicities dj. The dj +---+d, =d coefficients of the fs are the unknowns in the 
system of d linear equations a, = fi(n) rf +---+fi(n)rz (n=0,1,...,d—1), which 
has a unique solution. 


(1 > 3) Read off g(x) = 1+c,x+---+cyx@ from the recurrence; the coefficients of 
p(x) are [x*]p(x) = ag +c1ay_1 +++» +cgax_g for 0 <k < d, where a; = 0 fori <0. 
(2 > 1) Compute the c;s using g(x) = [Ik — rx) eft! 


(2 > 3) Let g(x) = J],(1 — rjx)*°2*!, and compute the first k terms of p(x) = 
A(x)q(x); the others are 0. 


(3 = 1) Extract the cjs from the denominator q(x). 
(3 = 2) Compute the partial fraction decomposition p(x) /q(x) = LV, pi(x)/(1 — 


rx)“ where deg p;(x) < d; and use (1 —rjx)-4% =Y, (ere to extract ad, = 
x" |p(x)/q(x). 


Characterizing polynomials. As a special case of Theorem 1.3.5, we obtain a 
useful characterization of sequences given by a polynomial. The difference operator 
A acts on sequences, sending the sequence {a, : n € N} to the sequence {Aa, : n € 
N} where Ady = an+1 — n- 


Theorem 1.3.6 [194, Theorem 4.1.1] Let ao,a,,a2,... be a sequence of complex 
numbers and let A(x) = Lys 4nx" be its ordinary generating function. Let d be a 
positive integer. The following are equivalent: 


1. We have At*!a, =0 for alln EN. 
2. There exists a polynomial f (x) with deg f <d such that a, = f(n) for alln EN. 


3. There exists a polynomial p(x) with deg p(x) < d such that A(x) = 
p(x)/(1—x)*". 


We have already seen some combinatorial polynomials and generating functions 
whose denominator is a power of 1 — x; we will see many more examples in the 
following sections. 
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1.3.4.2 Algebraic and D-finite generating functions 


When the generating function A(x) = Y),, anx" we are studying is not rational, the next 
natural question to ask is whether A(x) is algebraic. If it is, then just as in the rational 
case, the sequence a, still satisfies a linear recurrence, although now the coefficients 
are polynomial in n. This general phenomenon is best explained by introducing the 
wider family of “D-finite” (also known as “differentially finite” or “holonomic’”) 
power series. Let us discuss a quick example before we proceed to the general theory. 
We saw that the ordinary generating function for the Motzkin numbers satisfies 

the quadratic equation 
°M? +(x—-1)M+1=0 (1.8) 


which gives rise to the quadratic recurrence M, = M,_; +¥;MjM,_2_-; with Mo = 1. 
This is not a bad recurrence, but we can find a better one. Differentiating (1.8) we 
can express M’ in terms of M. Our likely first attempt leads us to M’ = —(2xM? + 
M) /(2x*M +x—1), which is not terribly enlightening. However, using (1.8) and a bit 
of purposeful algebraic manipulation, we can rewrite this as a linear equation with 
polynomial coefficients: 


(x — 2x? — 3x3)M! + (2 —3x—3x7)M—2=0. 
Extracting the coefficient of x” we obtain the much more efficient recurrence relation 
(n+2)M, — (2n+1)M,_1 — (3n—3)M,-2 = 0. (n > 2) 
We now explain the theoretical framework behind this example. 


Rational, algebraic, and D-finite series. Consider a formal power series A(x) over 
the complex numbers. We make the following definitions. 


A(x) is rational | There exist polynomials p(x) and q(x) 4 0 such that 


A(x) is algebraic | There exist polynomials po(x),..., pa(x) such that 


Po(x) + pi(x)A(x) + pa(x)A(x)? + +--+ pa(x)A(x)4 = 0. 


A(x) is D-finite | There exist polynomials go(x),...,qa(x),q(x) such that 


qo(x)A(x) +41 (x)A"(x) +92(2)A" (x) +--+ ga(xJA (x) = q(x). 


Now consider the corresponding sequence do,a),d2... and make the following 
definitions. 
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{ag,a,,..-} is | There are constants co,...,¢q € C such that for alln > d 


c-recursive 
Can + C1 An—1 +++ + Cgan—g = 0 


{ao,a),...} is | There are complex polynomials co(x),...,cqa(x) such that for 
P-recursive aln>d 


co(n)dn +1 (n)dn_1 +--+ +cg(n)adn_a = 0 


These families contain most (but certainly not all) series and sequences that we 
encounter in combinatorics. They are related as follows. 


Theorem 1.3.7 Let A(x) = ao +.a1x +a2x? +--+ be a formal power series. The fol- 
lowing implications hold. 


A(x) is rational => A(x) is algebraic => A(x) is D-finite 
q q 
{ao,a1,...} is c-recursive = {ao,a1,...} is P-recursive 


Proof. | We already discussed the correspondence between rational series and c- 
recursive functions, and rational series are trivially algebraic. Let us prove the re- 
maining statements. 


(Algebraic = D-finite) Suppose A(x) satisfies an algebraic equation of degree d. 
Then A is algebraic over the field C(x), and the field extension C(x,A) is a vector 
space over C(x) having dimension of at most d. 

Taking the derivative of the polynomial equation satisfied by A, we get an ex- 
pression for A’ as a rational function of A and x. Taking derivatives repeatedly, we 
find that all derivatives of A are in C(x,A). It follows that 1,4,A’,A”,...,A are 
linearly dependent over C(x), and a linear relation between them is a certificate for 
the D-finiteness of A. 


(P-recursive & D-finite) If qo(x)A(x) +q1(x)A'(x) +--+ qa(x)A (x) = q(x), com- 
paring the coefficients of x” gives a P-recursion for the a;s. In the other direction, 
given a P-recursion for the a;s of the form co(n)dy, +-+--+cg(n)dy_a = 0, it is easy 
to obtain the corresponding differential equation after writing c;(x) in terms of the 
basis {(x+i), : k © N} of C[x], where (y), = y(y—1)---(y—k +1). a 


The converses are not true. For instance, \/ 1 +. is algebraic but not rational, and 
e* and log(1 — x) are D-finite but not algebraic. 


Corollary 1.3.8 The ordinary generating function Y.,anx" is D-finite if and only if 
the exponential generating function Y., Gn™ is D-finite. 
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Proof. This follows from the observation that {a, : n © N} is P-recursive if and 
only if {a,/n! : n € N} is P-recursive. a 


A few examples. Before we discuss general tools, we collect some examples. We 
will prove all of the following statements later in this section. 

The power series for subsets, Fibonacci numbers, and Stirling numbers are ratio- 
nal: 


ee 1 i 
Ee saree dio = 


n>0 


X Xx Xx Xx 


S(n.k)x" = ; a ; 
BS aa ae ae 


n>k 


XX 


The “diagonal binomial,’ k-Catalan, and Motzkin series are algebraic but not 


rational: 
y (*") vl . 1 @e 
so\n 1—4x iso (k-1)nt+1 n 
and 
ym Fs 1—x-—vV1—2x—3x? 
nx = . 
D0 2x2 


The following series are D-finite but not algebraic: 


2n\* 3 
e*, log(l+x), sinx, cosx, arctanx, ¥ (7) x", sf , )y 
n>0 


The following series are not D-finite: 


1 
/1+log(1+x7), secx, tanx, )" p(n)x"=[] i 
—x 


n>0 k>0 


Recognizing algebraic and D-finite series. It is not always obvious whether a 
given power series is algebraic or D-finite, but there are some tools available. Fortu- 
nately, algebraic functions behave well under a few operations, and D-finite functions 
behave even better. This explains why these families contain most examples arising 
in combinatorics. 

The following table summarizes the properties of formal power series that are 
preserved under various key operations. For example, the fifth entry on the bottom 
row says that if A(x) and B(x) are D-finite, then the composition A(B(x)) is not 
necessarily D-finite. 


cA | A+B |AB/]1/A|AoB/AxB] A’ | [A] ATP 
rational Y Y lites Y Y | Y/N N 
algebraic | Y Y Y Y Y N |Y|]N Y 
D-finite | Y Y Y|N N Y |Y/Y N 
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Here A x B(x) := Ly504nbnx" denotes the Hadamard product of A(x) and B(x), 
JA(x) = Lns1 “1x” is the formal integral of A(x), and A‘-!)(x) is the composi- 
tional inverse of A(x). In the fourth column we are assuming that A(0) 4 0 so that 
1/A(x) is well-defined, in the fifth column we are assuming that B(0) = 0 so that 
A(B(x)) is well-defined, and in the last column we are assuming that A(0) = 0 and 
A’(0) £0, so that A‘—!) (x) is well-defined. 

For proofs of the “Yes” entries, see [183], [192], and [79]. For the “No” entries, 
we momentarily assume the statements of the previous subsection. Then we have the 
following counterexamples: 


e cosx is D-finite but 1/cosx = secx is not. 


e V1+x and log(1 +x’) are D-finite but their composition \/1 + log(1 +x?) 
is not. 


e A(x) =Lrs0 (2) is algebraic but AxA(x) = V5 2) x" is not. 
e 1/(1+ x) is rational and algebraic but its integral log(1 +x) is neither. 
e x+-.x? is rational but its compositional inverse (—1 + ./1+4x) /2 is not. 


e arctanx is D-finite but its compositional inverse tan is not. 
Some of these negative results have weaker positive counterparts: 
e If A(x) is algebraic and B(x) is rational, then A(x) x B(x) is algebraic. 


e If A(x) is D-finite and A(0) 4 0, 1/A(x) is D-finite if and only if A’ (x)/A(x) 
is algebraic. 


e If A(x) is D-finite and B(x) is algebraic with B(O) = 0, then A(B(x)) is D- 
finite. 


See [192, Proposition 6.1.11], [96], and [192, Theorem 6.4.10] for the respective 
proofs. 
The following result is also useful. 


Theorem 1.3.9 [192, Section 6.3] Consider a multivariate formal power series 


F(x1,...,Xq) that is rational in x,,...,xq and its diagonal: 
n Nn, : 
F(x1,..-,;%¢) = y Oni cimghy hey diag F(x) = y n,....nX". 
N1,...Ng>O n>0 


1. If d =2, then diag F (x) is algebraic. 


2. If d > 2, then diag F (x) is D-finite but not necessarily algebraic. 
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Now we are ready to prove our positive claims about the series at the beginning 
of this section. The first three expressions are visibly rational, and the diagonal bi- 
nomial and Motzkin series are visibly algebraic. We proved that the k-Catalan series 
is algebraic in Section 1.3.2.2. The functions e*, log(1 +x), sinx, cosx, arctanx sat- 
isfy the differential equations y’ = y, (1+x)y’ = 1, y” =—y, y” =—y, (14+27)y' = 1, 


respectively. The series Y,>9 (7") >" is the Hadamard product of (1 —4x)~!/? with 
itself, and hence D-finite. Finally, Y°,>0 eons - 


c 1 = atbt+c) va, boc ‘ 
ton = Yap.c>o ( abe )x y’z°, and hence D-finite. 


Proving the negative claims requires more effort and, often, a bit of analytic ma- 
chinery. We briefly outline some key results. 


‘eo is the diagonal of the rational func- 


Recognizing series that are not algebraic. There are a few methods available to 
prove that a series is not algebraic. The simplest algebraic and analytic criteria are 
the following. 


Theorem 1.3.10 (Eisenstein’s theorem [155]) [fa series A(x) =Yys04nx" with ra- 
tional coefficients is algebraic, then there exists a positive integer m such that a,m” 
is an integer for alln > 0. 


This shows that e*,log(1 +x), sin.x,cos.x, and arctan are not algebraic. 


Theorem 1.3.11 [111] If the coefficients of an algebraic power series A(x) = 
Ln>04nx" satisfy ay ~ cn'a" for nonzero c,a € C and r <0, then r cannot be a 
negative integer. 

Stirling’s approximation n! ~ V27n(n/e)" gives Cy ~c-16"/n and Gr) ~ 
c-27"/n, so the corresponding series are not algebraic. - 

Another useful analytic criterion is that an algebraic series A(x) must have a 
Newton-Puiseux expansion at any of its singularities. See [79, Theorem VII.7] and 
[78] for details. 


Recognizing series that are not D-finite. The most effective methods to show that 
a function is not D-finite are analytic. 


Theorem 1.3.12 [97, Theorem 9.1] Suppose that A(x) is analytic at x = 0, and it is 
D-finite, satisfying the equation qo(x)A(x) + 41(x)A'(x) +: +qa(x)A (x) = (x) 
with qa(x) #0. Then A(x) can be extended to an analytic function in any simply 
connected region of the complex plane not containing the (finitely many) zeroes of 
qa(x). 


Since secx and tanx have a pole at every odd multiple of 7, they are not D-finite. 
Similarly, ¥), p(n)x" = The, + is not D-finite because it has the circle |x| = 1 asa 
natural boundary of analyticity. 
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There are other powerful analytic criteria to prove a series is not D-finite. See 
[78, Theorem VII.7] for details and further examples. 

Sometimes it is possible to give ad hoc proofs that series are D-finite. For in- 
stance, consider y = \/1 +log(1+.x2). By induction, for any k € N there exist poly- 
nomials rj (x),...,7%(x) such that y®) = ry /y +r2/y? +++» +r;,/y’*!. An equation 
of the form Y49qi(x)y) = q(x) would then give rise to a polynomial equation sat- 
isfied by y. This would also make y* — 1 = log(1 +x”) algebraic; but this contradicts 
Theorem 1.3.10. 


1.4 Linear algebra methods 


There are several important theorems in enumerative combinatorics that express a 
combinatorial quantity in terms of a determinant. Of course, evaluating a determinant 
is not always straightforward, but there is a wide array of tools at our disposal. 

The goal of Section 1.4.1 is to reduce many combinatorial problems to “just com- 
puting a determinant”; examples include walks in a graph, spanning trees, Eulerian 
cycles, matchings, and routings. In particular, we discuss the transfer matrix method, 
which allows us to encode many combinatorial objects as walks in graphs, so that 
these linear algebraic tools apply. These problems lead us to many beautiful, myste- 
rious, and highly non-trivial determinantal evaluations. We will postpone the proofs 
of the evaluations until Section 1.4.2, which is an exposition of some of the main 
techniques in the subtle science of computing combinatorial determinants. 


1.4.1 Determinants in combinatorics 


1.4.1.1 Preliminaries: Graph matrices 


An undirected graph, or simply a graph G = (V,£) consists of a set V of vertices 
and a set E of edges {u,v} where u,v € V and u # v. In an undirected graph, we 
write uv for the edge {u,v}. The degree of a vertex is the number of edges incident 
to it. A walk is a set of edges of the form v1; v2, v2v3,...,Vx_1 Vg. This walk is closed 
if y= Vy. 

A directed graph or digraph G = (V,E) consists of a set V of vertices and a set 
E of oriented edges (u,v) where u,v € V and u ¥ v. In an undirected graph, we write 
uv for the directed edge (u,v). The outdegree (respectively, indegree) of a vertex is 
the number of edges coming out of it (respectively, coming into it). A walk is a set 
of directed edges of the form v1 v2, V2v3,...,Vg_1Vx- This walk is closed if vz = v1. 

We will see in this section that many graph theory problems can be solved using 
tools from linear algebra. There are several matrices associated to graphs that play a 
crucial role; we review them here. 
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Directed graphs. Let G = (V,E) be a directed graph. 
e The adjacency matrix A = A(G) is the V x V matrix whose entries are 


yy = number of edges from u to v. 


e The incidence matrix M = M(G) is the V x E matrix with 


1 if v is the final vertex of edge e, 
Mye= 4 —1 if vis the initial vertex of edge e, 
0 otherwise. 


> > 
e The directed Laplacian matrix L = L(G) is the V x V matrix whose en- 
tries are 


— J —(number of edges fromutov) ifufv, 
| outdeg(w) ifu=v. 


Undirected graphs. Let G = (V,E) be an undirected graph. 


e The (undirected) adjacency matrix A = A(G) is the V x V matrix whose 
entries are 
yy = number of edges connecting u and v. 


This is the directed adjacency matrix of the directed graph on V containing 
edges u — v and v — u for every edge uv of G. 


e The (undirected) Laplacian matrix L = L(G) is the V x V matrix with entries 


fis ae of edges connecting uandv) ifuA#v 


degu ifu=v 


If M is the incidence matrix of any orientation of the edges of G, then L = 
MM". 


1.4.1.2 Walks: the transfer matrix method 


Counting walks in a graph is a fundamental problem, which (often in disguise) in- 
cludes many important enumerative problems. The transfer matrix method addresses 
this problem by expressing the number of walks in a graph G in terms of its adjacency 
matrix A(G), and then uses linear algebra to count those walks. 
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Directed or undirected graphs. The transfer matrix method is based on the 
following simple, powerful observation, which applies to directed and undirected 
graphs: 


Theorem 1.4.1 Let G = (V,E) be a graph and let A= A(G) be the V x V adjacency 
matrix of G, where ayy, is the number of edges from u to v. Then 


(A”) uy = number of walks of length n in G from u to v. 
Proof. Observe that 


n 
(A Juv = y Quw, Aww *** Awy_ pv 


and there are dyy,Gw,w)***@w,_;v Walks of length n from u to v visiting vertices 
Uu,W1,---,;Wn—1,V in that order. a 


Corollary 1.4.2 The generating function Y,>09(A") wx" for walks of length n from u 
to v in Gis a rational function. 
Proof. Using Cramer’s formula, we have 


u+ydet(I—xA : v,u) 
det (I — xA) 


YA" )wx" = (A) = (1) 


n>0 


where (M : v,u) is the cofactor of M obtained by removing row v andcolumnu. 


Corollary 1.4.3 If Cg(n) is the number of closed walks of length n in G, then 


Ce(n) = Atte. +g, es 7 ee 


where A,,...,A, are the eigenvalues of adjacency matrix A and Q(x) = det(I—xA). 


Proof. Theorem 1.4.1 implies that Cg(n) = tr(A”) = Aj’ +---+ Aj. The second 
equation then follows from Q(x) = (1 — Ayx)--- (1 — Agx). a 


In view of Theorem 1.4.1, we want to be able to compute powers of the adjacency 
matrix A. As we learn in linear algebra, this is very easy to do if we are able to 
diagonalize A. This is not always possible, but we can do it when A is undirected. 


Undirected graphs. When our graph G is undirected, the adjacency matrix A(G) 
is symmetric, and hence diagonalizable. 


Theorem 1.4.4 Let G = (V,E) be an undirected graph and let ’,,...,A, be the 
eigenvalues of the adjacency matrix A = A(G). Then for any vertices u and v there 
exist constants C,,...,C such that 


number of walks of length n from u to v = cy Aj +--+ +c,df. 
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Proof. — The key fact is that a real symmetric k x k matrix A has k real orthonormal 
eigenvectors q1,...,q, with real eigenvalues A;,...,A,. Equivalently, the k x k matrix 
Q with columns q1,...,g, is orthogonal (so Q? = Q~!) and diagonalizes A: 


Q-'AQ = D = diag(Ay,..., Ax) 


where D = diag(A,,...,A,) is the diagonal matrix with diagonal entries A,,...,Ax,. 
The result then follows from A” = QD"Q-! = Odiag(A?,...,A7)Q7, with c, = 


Git jt- |_| 


Applications. Many families of combinatorial objects can be enumerated by first 
recasting the objects as walks in a “transfer graph” and then applying the transfer 
matrix method. We illustrate this technique with a few examples. 


1. (Colored necklaces) Let f(n,k) be the number of ways of coloring the beads 
of a necklace of length n with k colors so that no two adjacent beads have the 
same color. (Different rotations and reflections of a coloring are considered 
different.) There are several ways to compute this number, but a very efficient 
one is to notice that such a coloring is a graph walk in disguise. If we label the 
beads 1,...,” in clockwise order and let a; be the color of the ith bead, then 
the coloring corresponds to the closed walk aj,a2,...,@,,a, in the complete 
graph K;,. The adjacency graph of K, is A = J —I where J is the matrix all of 
whose entries equal 1, and J is the identity. Since J has rank 1, it has n— 1 
eigenvalues equal to 0. Since the trace is n, the last eigenvalue is n. It follows 
that the eigenvalues of A = J —J are —1,—1,...,—1,n— 1. Then Corollary 
1.4.3 tells us that 


f(n,k) = (n—1)§ + (n—1)(-1)*. 


It is possible to give a bijective proof of this formula, but this algebraic proof 
is much simpler. 


2. (Words with forbidden subwords, 1) Let h, be the number of words of length 
n in the alphabet {a,b} that do not contain aa as a consecutive subword. This 
is the same as a walk of length n — | in the transfer graph with vertices a and b 
and edges a — b, b+ a and b — b. The absence of the edge a —+ a guarantees 
that these walks produce only the valid words we wish to count. The adjacency 
matrix and its powers are 


= 0 1 n_ Fn-1 Fi 
a=({ iy are (Pe Ag) 
where the Fibonacci numbers Fo, F,... are defined recursively by Fo = 0, Fj = 


1, and Fh = Fy_1 + Fy_2 for k > 2. 


Since h, is the sum of the entries of A”~!, we get that h, = F,.2, and gy ~ c- Qt” 
where @ = 5(1 +5) © 1.6179... is the golden ratio. Of course there are eas- 
ier proofs of this fact, but this approach works for any problem of enumerating 
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words in a given alphabet with given forbidden consecutive subwords. Let us 
study a slightly more intricate example, which should make it clear how to 
proceed in general. 


3. (Words with forbidden subwords, 2) Let g, be the number of cyclic words of 
length n in the alphabet {a,b} that do not contain aa or abba as a consecutive 
subword. We wish to model these words as walks in a directed graph. At first 
this may seem impossible because, as we construct the word sequentially, the 
validity of a new letter depends on more than just the previous letter. However, 
a simple trick resolves this difficulty: We can introduce more memory into the 
vertices of the transfer graph. In this case, since the validity of a new letter 
depends on the previous three letters, we let the vertices of the transfer graph 
be aba,abb, bab, bba, bbb (the allowable “windows” of length 3) and put an 
edge wxy — xyz in the graph if the window wxy is allowed to precede the 
window xyz; that is, if wxyz is an allowed subword. The result is the graph of 
Figure 1.14, whose adjacency matrix A satisfies 


1 0 -x O 0 
0 1 0 0. =x 
det(I—xA)=det] —x -x 1 0 0. Saat ey ae ae 
—x 1 


alob 
ae ° 
aba bab bbb) 
YS 
bba 


Figure 1.14 
The transfer graph for words on the alphabet {a,b} avoiding aa and abba as consec- 
utive subwords. 


The valid cyclic words of length n correspond to the closed walks of length 
n in the transfer graph, so Corollary 1.4.3 tells us that the generating function 
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for gy is 

x+2x? —3x3 +4x4 
Y nx" = 24.43 34 
0 l-x-—x° +x? -x 


= x43x7 499 +7x4 + 6° + 15x° 4 15x? + 31x89 437209 ++. 


Theorem 1.3.5.2 then tells us that g, ~ c- a@” where & © 1.5129 is the inverse 
of the smallest positive root of 1 — x —x* +x? —x* =0. The values of g1, 22,23 
may surprise us. Note that the generating function does something counterin- 
tuitive: it does not count the words a (because aa is forbidden), aba (because 
aa is forbidden), or abb (because abba is forbidden). 


This example serves as a word of caution: When we use the transfer matrix 


method to enumerate “cyclic” objects using Corollary 1.4.3, the initial values of 
the generating function may not be the ones we expect. In a particular problem of 
interest, it will be straightforward to adjust those values accordingly. 


To illustrate the wide applicability of this method, we conclude this section with 


a problem where the transfer graph is less apparent. 


4. (Monomer-dimer problem) An important open problem in statistical mechan- 


ics is the monomer-dimer problem of computing the number of tilings 
T (m,n) of anm xn rectangle into dominoes (2 x 1 rectangles) and unit squares. 
Equivalently, T(m,n) is the number of partial matchings of an m x n grid, 
where each node is matched to at most one of its neighbors. 


There is experimental evidence, but no proof, that T(n,n) ~c- a” where ot © 
1.9402... is a constant for which no exact expression is known. The transfer- 
matrix method is able to solve this problem for any fixed value of m, proving 
that the generating function Y,59 T(m,n)x" is rational. We carry this out for 
m=3. 


Let t(n) be the number of tilings of a 3 x n rectangle into dominoes and unit 
squares. As with words, we can build our tilings sequentially from left to right 
by covering the first column, then the second column, and so on. The tiles 
that we can place on a new column depend only on the tiles from the previous 
column that are sticking out, and this can be modeled by a transfer graph. 


More specifically, let T be a tiling of a 3 x n rectangle. We define n+ 1 
triples vo,...,V¥n which record how T interacts with the n+ 1 vertical grid 
lines of the rectangle. The ith grid line consists of three unit segments, and 
each coordinate of v; is 0 or 1 depending on whether these three segments are 
edges of the tiling or not. For example, Figure 1.15 corresponds to the triples 
111,110,011, 101,010,111. 


The choice of v; is restricted only by vj_;. The only restriction is that v;_; and 
v; cannot both have a 0 in the same position, because this would force us to put 
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Pea Ey He ee 


Figure 1.15 
A tiling of a 3 x 5 rectangle into dominoes and unit squares. 


two overlapping horizontal dominoes in T. These compatibility conditions are 
recorded in the transfer graph of Figure 1.16. When v;_; = v; = 111, there are 
three ways of covering column i. If v;_; and v; share two 1s in consecutive po- 
sitions, there are two ways. In all other cases, there is a unique way. It follows 
that the tilings of a 3 x n rectangle are in bijection with the walks of length n 
from 111 to 111 in the transfer graph. 


an. 
oe 000 eS 


Figure 1.16 
The transfer graph for tilings of 3 x n rectangles into dominoes and unit squares. 
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Since the adjacency matrix is 


000 O01 110 O10 101 OO1 110 111 

000 / 0 0 0 0 0 0 0 1 

001 0 0 1 0 0 0 0 1 

110} 0 1 0 0 1 0 1 2 

oe 010} O 0 0 0 1 0 0 1 
101 0 0 1 1 0 0 1 1 

001 0 0 0 0 0 0 1 1 

110} 0 0 1 0 1 1 0 2 

111 1 1 2 1 1 1 2 3 


Theorem 1.4.1 tells us that 


det(—xA : 111,111) 
t(n)x” = 
die det(I —xA) 


n>0 


(1+x—x7)(1—2x—x?) 
(1 +x)(1 —5x— 9x? + 9x3 4.24 — x5) 
= 143x+22x7 + 1312? +8234 + 5096x° + 31687x° 4+ --- 


By Theorem 1.3.5.2, th ~ c- @” where @ * 6.21207... is the inverse of the 
smallest positive root of the denominator (1 +x) (1 —5x—9x* + 9x3 +x4 — x9). 


1.4.1.3 Spanning trees: the matrix-tree theorem 


In this section we discuss two results: Kirkhoff’s determinantal formula for the num- 
ber of spanning trees of a graph, and Tutte’s generalization to oriented spanning trees 
of directed graphs. 


Undirected matrix-tree theorem. Let G = (V,E) be a connected graph with no 
loops. A spanning tree 7 of G is a collection of edges such that for any two vertices 
u and v, T contains a unique path between u and v. If G has n vertices, then 


e T contains no cycles, 
e T spans G; that is, there is a path from u to v in T for any vertices u # v, and 
e Thasn—1 edges. 


Furthermore, any two of these properties imply that T is a spanning tree. Our goal in 
this section is to compute the number c(G) of spanning trees of G. 

Orient the edges of G arbitrarily. Recall that the incidence matrix M of G is the 
V x E matrix whose eth column is e, — e, if e = u — v, where e; is the ith basis 
vector. The Laplacian L = MMT has entries 


1 = —(number of edges connecting uandv) ifuf¢v 
ers deg u ifu=v 


Algebraic and Geometric Methods in Enumerative Combinatorics 51 


Note that L(G) is singular because all its row sums are 0. A principal cofactor L,(G) 
is obtained from L(G) by removing the vth row and vth column for some vertex v. 


Theorem 1.4.5 (Kirkhoff’s Matrix-Tree Theorem) The number c(G) of spanning 
trees of a connected graph G is 


c(G) =detL,(G) = a oy, ey 


where L(G) is any principal cofactor of the Laplacian L(G), and Ay,...,An—1,An = 
0 are the eigenvalues of L(G). 


Proof. | We use the Binet-Cauchy formula, which states that if A and B arem xn 
and n x m matrices, respectively, with m <n, then 


detAB= )'  detA[S]detB(S] 
SC In] :|S|=m 


where A[S] (respectively, B[S]) is the n x n matrix obtained by considering only the 
columns of A (respectively, the rows of B) indexed by S. 

We also use the following observation: If M, is the “reduced” adjacency matrix 
M with the vth row removed, and S is a set of n — 1 edges of E, then 


det M, [S] = fa if S is a spanning tree, 


0 otherwise. 


This observation is easily proved: If S is not a spanning tree, then it contains a cycle 
C, which gives a linear dependence among the columns indexed by the edges of C. 
Otherwise, if S is a spanning tree, think of v as its root, and “prune” it by repeatedly 
removing a leaf v; 4 v and its only incident edge e; for 1 <i <n—1. Then if we list 
the rows and columns of M[S] in the orders v1,...,v,—1 and e1,...,€,—1, respectively, 
the matrix will be lower triangular with 1s and —1s in the diagonal. 

Combining these two equations, we obtain the first statement: 


detL,(G)= Y  detM[S|detM™|S]= YY  detM[S]? =c(G). 
SC In] : |S] =m SC [n] :|S|=m 


To prove the second one, observe that the coefficient of —x! in the characteristic 
polynomial det(Z— xJ) = (A; —x)-++ (An—1 — x)(0 — x) is the sum of the n principal 
cofactors, which are all equal to c(G). a 


The matrix-tree theorem is a very powerful tool for computing the number of 
spanning trees of a graph. Let us state a few examples. 

The complete graph K,, has n vertices and an edge joining each pair of vertices. 
The complete bipartite graph K,,,,, has m “top” vertices and n “bottom” vertices, 
and mn edges joining each top vertex to each bottom vertex. The hyperoctahedral 
graph ©), has vertices {1,1’,2,2’,...,n,n’} and its only missing edges are ii’ for 1 < 
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i <n. The n-cube graph C,, has vertices (€),...,€,) where €; € {0,1}, and an edge 
connecting any two vertices that differ in exactly one coordinate. The n-dimensional 
grid of size m, denoted mC,, has vertices (€1,...,€,) where €; € {1,...,m}, and an 
edge connecting any two vertices that differ in exactly one coordinate i, where they 
differ by 1. 


Theorem 1.4.6 The number of spanning trees of some interesting graphs are as fol- 
lows. 


1. (Complete graph) c(Kn) =n"? 


2. (Complete bipartite graph) C(Kmn) =m" 'n'"™! 


3. (Hyperoctahedral graph) c(Qn) = 27"~?(n—1)"n"~? 


4, (n-cube) c(Cy) = 22°"! yp, kM) 
5. (n-dimensional grid of size m) c(mC,) = m'™"~"—"T]"_, KCa)im—1)" 


We will see proofs of the first and third example in Section 1.4.2. For the others, 
and many additional examples, see [55]. 


Directed matrix-tree theorem. Now let G = (V,£) be a directed graph containing 
no loops. An oriented spanning tree rooted at v is a collection of edges T such that 
for any vertex u there is a unique path from u to v. The underlying unoriented graph 
T is a spanning tree of the unoriented graph G. Let c(G, v) be the number of spanning 
trees rooted at G. = 

Recall that the directed Laplacian matrix L has entries 


— _ |—(number of edges fromu tov) ifu Av 
7 outdeg u ifu=v 


Now the matrix L (G) is not necessarily symmetric, but it is still singular. 


Theorem 1.4.7 (Tutte’s Directed Matrix-Tree Theorem) Let G be a directed 
graph and v be a vertex. The number c(G,v) of oriented spanning trees rooted at 
vis 
—> 
c(G,v) = det L(G) 
—> 

where L(G) is obtained from L(G) by removing the vth row and column. Further- 
more, if G is balanced, so indeg v = outdeg v for all vertices v, then 


c(G,v) = “ay _s An-1 


where A,,...,An—1, An = 0 are the eigenvalues of L(G). 
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Proof. Proceed by induction. Consider a vertex w 4 v and an edge e starting at w. 
Let G’ = G—e be obtained from G by removing e. If e is the only edge starting at w, 
then every spanning tree must use it, and we have 


c(G,v) = c(G',v) = det L(G’) = det L(G). 


Otherwise, let G” be obtained from G by removing all edges starting at w other than 
e. There are c(G’,v) oriented spanning trees rooted at v that do not contain e, and 
c(G",v) that do contain e, so we have 


e(G,v) = c(G’,v) +¢(G",v) = det L(G’) +-det L(G") = det L(G) 


where the last equality holds since determinants are multilinear. 
We postpone the proof of the second statement to the next section, where it will 
be an immediate consequence of Theorem 1.4.9. a 


1.4.1.4 Eulerian cycles: the BEST theorem 


One of the earliest combinatorial questions is the problem of the Seven Bridges of 
KOonigsberg. In the early 1700s, the Prussian city of K6nigsberg was separated by 
the Pregel river into four regions, connected to each other by seven bridges. In the 
map of Figure 1.17 we have labeled the regions NV,S,F, and J; there are two bridges 
between N and J, two between S and /, and three bridges connecting E to each of 
N,I, and S. The problem was to find a walk through the city that crossed each bridge 
exactly once. Euler proved in 1735 that it was impossible to find such a walk; this is 
considered to be the first paper in graph theory. 


Figure 1.17 
The seven bridges of K6nigsberg. Public domain map by Merian-Erben, 1652. 
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Euler’s argument is simple, and relies on the fact that every region of K6nigsberg 
is adjacent to an odd number of bridges. Suppose there existed such a walk, starting 
at region A and ending at region B. Now consider a region C other than A and B. 
Then our path would enter and leave C the same number of times; but then it would 
not use all the bridges adjacent to C, because there is an odd number of such bridges. 

In modern terminology, each region of the city is represented by a vertex, and 
each bridge is represented by an edge connecting two vertices. We will be more 
interested in the directed case, where every edge has an assigned direction. An Eu- 
lerian path is a path in the graph that visits every edge exactly once. If the path 
starts and ends at the same vertex, then it is called an Eulerian cycle. We say G is an 
Eulerian graph if it has an Eulerian cycle. 


Theorem 1.4.8 A directed graph is Eulerian if and only if it is connected and every 
vertex v satisfies indeg(v) = outdeg(v). 


Proof. \f a graph has an Eulerian cycle C, then C enters and leaves each vertex v 
the same number of times. Therefore indeg(v) = outdeg(v). 

To prove the converse, let us start by arbitrarily “walking around G until we get 
stuck.” More specifically, we start at any vertex vo, and at each step, we exit the 
current vertex by walking along any outgoing edge we have not used yet. If there is 
no available outgoing edge, we stop. 

Whenever we enter a vertex v 4 vo, we will also be able to exit it since indeg(v) = 
outdeg(v); so the walk can only get stuck at vo. Hence the resulting walk C is a cycle. 
If C uses all edges of the graph, we are done. If not, then since G is connected we 
can find a vertex v’ of C with an unused outgoing edge, and we use this edge to start 
walking around the graph G —C until we get stuck, necessarily at v’. The result will 
be a cycle C’. Starting at v’ we can traverse C and then C’, thus obtaining a cycle 
CUC' that is longer than C. Repeating this procedure, we will eventually construct 
an Eulerian cycle. a 


There is a remarkable formula for the number of Eulerian cycles, due to de Bruijn, 
van Ardenne-Ehrenfest, Smith, and Tutte. 


Theorem 1.4.9 (BEST Theorem) /f G is an Eulerian directed graph, then the num- 
ber of Eulerian cycles of G is 


c(G,v)- |] (outdeg(w) — 1)! 


weV 


for any vertex v, where c(G,v) is the number of oriented spanning trees rooted at v. 


Proof. We fix an edge e starting at v, and let each Eulerian cycle start at e. For each 
vertex w let E,, be the set of outgoing edges from w. 

Consider an Eulerian cycle C. For each vertex w # v, let ey be the last outgoing 
edge from w that C visits, and let 7,, (respectively, 7) be the ordered set Ey, — ey 
(respectively, E,,— e) of the other outgoing edges from w (respectively, v), listed in 
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the order that C traverses them. It is easy to see that T = {e,, : w Av} is an oriented 
spanning tree rooted at v. 

Conversely, an oriented tree T and permutations {7,, : w € V} serve as directions 
to tour G. We start with edge e. Each time we arrive at vertex w, we exit it by using 
the first unused edge according to 7,,. If we have used all the edges E,, — e,, of 1, 
then we use e,, € T. It is not hard to check that this is a bijection. This completes the 
proof. | 


Corollary 1.4.10 In an Eulerian directed graph, the number of oriented spanning 
trees rooted at v is the same for all vertices v; it equals 


1 
c(G,v) = ma ay “An-1 
. => 
where A,,...,An—1,An = 0 are the eigenvalues of L(G). 


Proof. |The BEST theorem implies that c(G,v) is independent of v, and then the 
argument in the proof of Theorem 1.4.5 applies to give the desired formula. a 


The BEST theorem can be used beautifully to enumerate a very classical, and 
highly nontrivial, family of objects. A k-ary de Bruijn sequence of order n is a 
cyclic word W of length k” in the alphabet {1,...,k} such that the k” consecutive 
subwords of W of length n are the k” distinct words of length n. For example, the 
2-ary deBruijn sequences of order 3 are 11121222 and 22212111; these “memory 
wheels” were described in Sanskrit poetry several centuries ago [112]. Their exis- 
tence and enumeration was proved by Flye Saint-Marie in 1894 for k = 2 and by van 
Aardenne-Ehrenfest and de Bruijn in 1951 in general. 


Theorem 1.4.11 [80, 58] The number of k-ary de Bruijn sequences of order n is 
n—1 
(kK!) /k". 


Proof. Consider the de Bruijn graph whose vertices are the k”~! sequences of 
length n— 1 in the alphabet {1,...,k}, and where there is an edge from a)a2...dy—1 
to the word a2a3...dy for all aj,...,ay. It is natural to label this edge ajaz...dy. It 
then becomes apparent that k-ary de Bruijn sequences are in bijection with the Eu- 
lerian cycles of the de Bruijn graph. Since indeg(v) = outdeg(v) = k for all vertices 
v, this graph is indeed Eulerian, and we proceed to count its Eulerian cycles. Notice 
that for any vertices u and v there is a unique path of length n from u to v. Therefore 
the k”—! x k"“! adjacency matrix A satisfies A” = J, where J is the matrix whose en- 
tries are all equal to 1. We already saw that the eigenvalues of J are 0,...,0,k”. Since 
the trace of A is k, the eigenvalues of A must be 0,...,0,k. Therefore the Laplacian 
L=kI—A has eigenvalues k,...,k,0. It follows from Corollary 1.4.10 that the de 
Bruijn graph has c(G,v) = ae 1 Ke! =1) — =" oriented spanning trees rooted at 
any vertex v, and 


c(G,v)- T] (outdeg(w) — 1)! = (ke 


weV 


Eulerian cycles, as desired. a 
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1.4.1.5 Perfect matchings: the Pfaffian method 


A perfect matching of a graph G = (V,£) is a set M of edges such that every vertex 
of Gis on exactly one edge from M. We are interested in computing the number m(G) 
of perfect matchings of a graph G. We cannot expect to be able to do this in general; 
in fact, even for bipartite graphs G, the problem of computing m(G) is #P-complete. 
However, for many graphs of interest, including all planar graphs, there is a beautiful 
technique that produces a determinantal formula for m(G). 


Determinants and Pfaffians. Let A be a skew-symmetric matrix of size 2m x 2m, 
so A? = —A. The Pfaffian is a polynomial encoding the matchings of the complete 
graph K>,,. A perfect matching M of the complete graph K>,, is a partition M of [2m] 
into disjoint pairs {i1, j1},..., {im, jm}, where i, < jy for 1 <k <m. Draw the points 
1,...,2m in order on a line and connect each i, to j, by a semicircle above the line. 
Let cr(M) be the number of crossings in this drawing, and let sign(M) = (—1)*™), 
Let ay = Qi j) °° Din jm: The Pfaffian of A is 


P£(A) =) sign(M ay 
M 


summing over all perfect matchings M of the complete graph K,,. 
Theorem 1.4.12 [fA is a skew-symmetric matrix, so A! =~—A, then 


det(A) = Pf(A)?. 


Sketch of Proof: The first step is to show that the skew symmetry of A causes many 
cancellations in the determinant, and 


detA = Ly sign(7)az 
NEECSp, 

where ECS, C S, is the set of permutations of [n] having only cycles of even 
length. Then, to prove that this equals (Yysign(M)ay)*, we need a bijection be- 
tween ordered pairs (M,,Mz) of matchings and permutations z in ECS, such that 
am, 4M, = az and sign(M )sign(M2) = sign(z). We now describe such a bijection. 

Draw the matchings M, and M) above and below the points 1,...,7 on a line, 
respectively. Let 2 be the permutation given by the cycles of the resulting graph, 
where each cycle is oriented following the direction of Mj at its smallest element. 
This is illustrated in Figure 1.18. It is clear that ay,ay, = az, while some care is 
required to show that sign(M, )sign(M2) = sign(z). For details, see [2]. a 


Counting perfect matchings via Pfaffians. Suppose we wish to compute the num- 
ber m(G) of perfect matchings of a graph G = (V,E) with no loops. After choosing 
an orientation of the edges, we define the V x V signed adjacency matrix S(G) 
whose entries are 
1 if i — 7 is an edge of G 
Sij= 4-1 if j-iis anedge of G 


0 otherwise. 
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GF 


Figure 1.18 

The pair of matchings {1,3}, {2,6}, {4,8},{5,7} and {1,5}, {2,4}, {3,7}, {6,8} 
gives the permutation (1375) (2684) with (—a 13426448457 )(—a15424037468) = 
4134377551426 A6gdg4a42. 


Then, for {i j1..-imjm} = {1,--.,2m}, Sm = Si; j,°**Sin jm 18 BONZero if and only if 
{i,j },---5 {ims jm} is a perfect matching of G. 

We say that our edge orientation is Pfaffian if all the perfect matchings of G 
have the same sign. At the moment there is no efficient test to determine whether 
a graph admits a Pfaffian orientation. There is a simple combinatorial restatement: 
An orientation is Pfaffian if and only if every even cycle C for which G\V(C) has a 
perfect matching has an odd number of edges in each direction. 

Fortunately, we have the following result of Kasteleyn [115, 131]: 


Every planar graph has a Pfaffian orientation. 
This is very desirable, because Theorem 1.4.12 implies the following: 
For a Pfaffian orientation of G, m(G) = \/detS(G). 


Therefore the number of matchings of a planar graph is reduced to the evaluation 
of a combinatorial determinant. We will see in Section 1.4.2 that there are many 
techniques at our disposal to carry out this evaluation. 

Let us illustrate this method with an important example, due to Kasteleyn [115] 
and Temperley—Fisher [198]. 


Theorem 1.4.13 The number m(R,») of matchings of the a x b rectangular grid Rap 
(where we assume b is even) is 


|a/2| b/2 a2 wk 
m(Rap) = 4le/2](/2) I I] (cos 7 J + cos? ) ~ cet w C-1,3385%, 


na 

I 
en 
> 

Il 
aR 


pom 1 ° ’ 
where G=1— gt+as—gpotcc is Catalan’s constant. 


Clearly this is also the number of domino tilings of an a x b rectangle. 
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Sketch of Proof. Orient all columns of Rz» going up, and let the rows alternate 
between going right or left, assigning the same direction to all edges of the same 
row. The resulting orientation is Pfaffian because every square has an odd number 
of edges in each direction. The adjacency matrix S satisfies m(Ra,) = VdetS. To 
compute this determinant, it is slightly easier * to consider the following mn x mn 
matrix B: 

1. ifiand j are horizontal neighbors 

bij =i if iand j are vertical neighbors 
0 otherwise. 


We can obtain B from the S by scaling the rows and columns by suitable powers 
of i, so we still have m(Ra») = \/|detB|. We will prove the product rule for this 
determinant in Section 1.4.2. 

We then use this product formula to give an asymptotic formula for m(R,»). Note 
that logm(R,»)/ab may be regarded as a Riemann sum; as m,n — © it converges to 


1 


C= 7, 
72 


u/2 pa/2 G 
[ log(4cos” x +. 4cos? y) dxdy = = 
0 JO 


where G is Catalan’s constant. Therefore 
m(Rap) © et” 1.3385”, 


Loosely speaking, this means that in a matching of the rectangular grid there are 
about 1.3385 degrees of freedom per vertex. | 


Obviously, this beautiful formula is not an efficient method of computing the 
exact value of m(R,») for particular values of a and b; it is not even clear why it 
gives an integer! There are alternative determinantal formulas for this quantity that 
are more tractable; see for example [2, Section 10.1]. 


1.4.1.6 Routings: the Lindstrom-—Gessel-Viennot lemma 


Let G be a directed graph with no directed cycles, which has a weight wt(e) on each 
edge e. We are most often interested in the unweighted case, where all weights are 
1. Let S = {51,...,5,} and T = {t1,...,t,} be two (not necessarily disjoint) sets of 
vertices, which we call sources and sinks, respectively. A routing from S to T is a set 
of paths P|,...,P, from the n sources 51,...,5, to the n sinks t),...,t, such that no 
two paths share a vertex. Let 2 be the permutation of [n] such that P; starts at source 
s; and ends at sink f,(;, and define sign(R) = sign(7). 

Let the weight of a path or a routing be the product of the weights of the edges it 
contains. Consider the n x n path matrix Q whose (i, /) entry is 


qij = y wt(P). 


P path from 5; to t; 


*TIn fact, this is the matrix that Kasteleyn uses in his computation. 


Algebraic and Geometric Methods in Enumerative Combinatorics 59 


Theorem 1.4.14 (Lindstr6m—Gessel—Viennot lemma) Let G be a directed acyclic 
graph with edge weights, and let S = {s1,...,8,} and T = {t,...,tn} be sets of 
vertices in G. Then the determinant of the n x n path matrix Q is 


detQ = y sign(R)wt(R). 


R routing from S to T 
In particular, if all edge weights are | and if every routing takes s; to t; for all i, then 


detQ = number of routings from S to T. 


Proof. We have detA = Ypsign(P)wt(P) summing over all path systems P = 
{P,,...,P,} from S to T; we need to cancel out the path systems that are not rout- 
ings. For each such P, consider the lexicographically first pair of paths P; and P; that 
intersect, and let v be their first vertex of intersection. Now exchange the subpath of 
P, from s; to v and the subpath of P; from s; to v, to obtain new paths P! and P*. 
Replacing {P;,P;} with {P/,P'}, we obtain a new path system @(P) from S to T. 
Notice that @(@(P)) = P, and sign(@(P))wt(@(P)) + sign(P)wt(P) = 0; so for all 
non-routings P, the path systems P and @(P) cancel each other out. | 


This theorem was also anticipated by Karlin and McGregor [114] in the context 
of birth-and-death Markov processes. 


Determinants via routings. The Lindstrém-Gessel-Viennot lemma is also a useful 
combinatorial tool for computing determinants of interest, usually by enumerating 
routings in a lattice. We illustrate this with several examples. 


1. (Binomial determinants) Consider the binomial determinant 


(rs) -mlG) 
by,..-,Dn bj] \i<ij<n 


where 0 < aj <-:-<a,and0 <b; <--- <b, are integers. These determinants 
arise as coefficients of the Chern class of the tensor product of two vector 
bundles. [129] This algebro-geometric interpretation implies these numbers 
are positive integers; as combinatorialists, we would like to know what they 
count. 


ASE path is a lattice path in the square lattice N? consisting of unit steps south 
and east. Consider the sets of points A = {Aj,...,An} and B = {Bj,...,Bn} 
where A; = (0,a;) and B; = (b;,b;) for 1 <i <n. Since there are (si) SE paths 
from A; to B;, and since every SE routing from A to B takes A; to B; for all i, 
we have 


a1,---,4n = number of SE routings from A to B. 
by,...,Dn 


This is the setting in which Gessel and Viennot discovered Theorem 1.4.14; 
they also evaluated these determinants in several special cases. [88] We now 
discuss one particularly interesting special case. 
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2. (Counting permutations by descent set) The descent set of a permutation 7 is 


the set of indices i such that 7; > 741. We now prove that 


Cly.+ ey Ck ; 
€ ee ) = number of permutations of [n] with descent set {c1,...,cx}, 
rC1,+++5Ck 


for any 0 < cy <-+-<cy <n. It is useful to define co = 0,cg41 =n. 


Encode such a permutation 7 by a routing as follows. For each i let f; be the 
number of indices j <i such that 7; < 7;. Note that the descents cy,...,c; of 
m are the positions where f does not increase. Splitting f at these positions, 
we are left with k+ 1 increasing subwords f!,..., f*+!. Now, for 1 <i<n+1 
let P; be the NW path from (cj_1,c;-1) to (0,c;) taking steps north precisely at 
the steps listed in f’. These paths give one of the routings enumerated by the 
binomial determinant in question, and this is a bijection. See Figure 1.19 for 
an illustration. [88] 


Figure 1.19 
The routing corresponding to 7 = 28351674 and f(m) = 12.23.157.4. 


3. (Rhombus tilings and plane partitions) Let R, be the number of tilings of a 


regular hexagon of side length n using unit rhombi with angles 60° and 120°. 
Their enumeration is due to MacMahon [133]. There are several equivalent 
combinatorial models for this problem, illustrated in Figure 1.20, which we 
now discuss. 


Firstly, it is almost inevitable to view these tilings as three-dimensional pic- 
tures. This shows that R, is also the number of ways of stacking unit cubes into 
the corner of a cubical box of side length n. Incidentally, this three-dimensional 
view makes it apparent that there are exactly n* rhombi of each one of the three 
possible orientations. 


Secondly, we may consider the triangular grid inside our hexagon, and place 
a dot on the center of each triangle. These dots form a hexagonal grid, where 
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Figure 1.20 
Four models for the rhombus tilings of a hexagon. 


two dots are neighbors if they are at distance | from each other. Finally, join 
two neighboring dots when the corresponding triangles are covered by a tile. 
The result is a perfect matching of the hexagonal grid. 


Next, on each one of the n2 squares of the floor of the box, write down the 
number of cubes above it. The result is a plane partition: an array of non- 
negative integers (finitely many of which are non-zero) that is weakly decreas- 
ing in each row and column. We conclude that R,, is also the number of plane 
partitions whose non-zero entries are at most n, and fit inside an n x n square. 


Finally, given such a rhombus tiling, construct n paths as follows. Each path 
starts at the center of one of the vertical edges on the western border of the 
hexagon, and successively crosses each tile splitting it into equal halves. It 
eventually comes out at the southeast side of the diamond, at the same height 
where it started (as is apparent from the 3-D picture). The final result is a 
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routing from the n sources S,,...,5, on the left to the sinks 7),...,7,, on the 
right in the “rhombus” graph shown below. It is clear how to recover the tiling 
from the routing. Since there are ( Fe 7) paths from §; to Tj, the Lindstr6m— 
Gessel—Viennot lemma tells us that R,, 1s given by the determinant 


2 ca ey ny oe | 
a=ael( 2 | = TT 
REI) | eigen ijk=I i+j+k—2 


We will prove this product formula in Section 1.4.2. 


. (Catalan determinants, multitriangulations, and Pfaffian rings) The Hankel 


matrices of a sequence A = (do,a1,a2,...) are 


ao a\ an 
a\ a2 st Gn+1 
Qn 4n+1 °"° a2n 
and 
a\ a2 Qn+1 
; a2 a3 an42 
H,,(A) = 
Qn4+1 4nt+2 °**  G2nt+1 


Note that if we know the Hankel determinants detH,,(A) and detH/(A) and 
they are nonzero for all n, then we can use them as a recurrence relation to 
recover each ax from ao,...,dx—1. 


There is a natural interpretation of the Hankel matrices of the Catalan sequence 
C = (Co,C,C2,...). Consider the “diagonal” grid on the upper half plane with 
steps (1,1) and (1,—1). Let A; = (—2i,0) and B; = (2,0). Then there are Cj; 
paths from A; to B;, and there is clearly a unique routing from (Ao,...,A;) to 
(Bo,.--,Bn). See Figure 1.21 for an illustration. This proves that det H,(C) = 1, 
and an analogous argument proves that det H/,(C) = 1. Therefore 


detH,,(A) = detH,,(A) =1foralln>0 <= Ais the Catalan sequence. 


The Hankel determinants of the shifted Catalan sequences also arise naturally 
in several contexts; they are given by: 


Cn-2k = Cn-2ket1 tt Crk 
i Cn—2k+1  Cn-2e42 + Cake i+ j+2(k—1) 
et : ; . ‘ = ee 
: : ; : it j<n—2k-1 ery 
Ch-k-1 Cn—k Sor Cn—2 


(1.9) 
There are several ways of proving this equality; for instance, it is a conse- 
quence of [122, Theorem 26]. We describe three appearances of this determi- 
nant. 
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Figure 1.21 


Routing interpretation of the Hankel determinant H,,(C). 


(a) A k-fan of Dyck paths of length 27 is a collection of k Dyck paths from 
(—n,0) to (n,0) that do not cross (although they necessarily share some 
edges). Shifting the (i+ 1)th path i units up and adding i upsteps at the 
beginning and i downsteps at the end, we obtain a routing of k Dyck paths 
starting at the points A = {—(n+k—1),...,—(n+1),—n} and ending at 
the points B = {n,n+1,...,n+k—1} on the x-axis. See Figure 1.22 
for an illustration. It follows that the number of k-fans of Dyck paths of 
length 2(n — 2k) is given by (1.9). 


VNVAYN 


Figure 1.22 
A k-fan of Dyck paths. 


(b) There is also an extension of the classical one-to-one correspondence be- 

tween Dyck paths and triangulations of a polygon. Define a k-crossing in 
an n-gon to be a set of k diagonals that cross pairwise. A k-triangulation 
is a maximal set of diagonals with no (k + 1)-crossings. The main 
enumerative result, due to Jonsson [110], is that the number of k- 
triangulations of an n-gon is also given by (1.9). A subtle bijection with 
fans of Dyck paths is given in [177]. 
Several properties of triangulations extend non-trivially to this context. 
For example, every k-triangulation has exactly k(2n — 2k — 1) diagonals 
[148, 69]. The k-triangulations are naturally the facets of a simplicial 
complex called the multiassociahedron, which is topologically a sphere 
[109]; it is not currently known whether it is polytopal. There is a fur- 
ther generalization in the context of Coxeter groups, with connections to 
cluster algebras [49]. 
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(c) These determinants also arise naturally in the commutative algebraic 
properties of Pfaffians, defined earlier in this section. Let A be a skew- 
symmetric n x n matrix whose entries above the diagonal are indetermi- 
nates {ajj : 1 <i< j <n} over a field k. Consider the Pfaffian ideal 
I,(A) generated by the (5;) Pfaffian minors of A of size 2k x 2k, and the 
Pfaffian ring R,(A) = k{aj;;]/1(A). Then the multiplicity of the Pfaffian 
ring R;,(X) is also given by (1.9). [99, 89] 


5. (Schroder determinants and Aztec diamonds) Recall from Section 1.3.2.2 that 


a Schréder path of length 7 is a path from (0,0) to (27,0) using steps NE = 
(1,1),SE = (1,-1), and E = (2,0) that stays above the x-axis. The Hankel 
determinant detH,,(R) counts the routings of Schréder paths from the points 
A= {0,—2,...,—(2n)} to the points B = {0,2,...,2n} on the x-axis. 


These Hankel determinants have a natural interpretation in terms of tilings. 
Consider the Aztec diamond * AD,, consisting of 21 rows centered horizon- 
tally, consisting successively of 2,4,...,2,2n,...,4,2 squares. We are inter- 
ested in counting the tilings of the Aztec diamond into dominoes. 


Figure 1.23 
A tiling of the Aztec diamond and the corresponding routing. 


Given a domino tiling of AD, construct n paths as follows. Each path starts 
at the center of one of the vertical unit edges on the southwest border of 
the diamond, and successively crosses each tile that it encounters following 
a straight line through the center of the tile. It eventually comes out at the 


*This shape is called the Aztec diamond because it is reminiscent of designs of several Native American 
groups. Perhaps the closest similarity is with Mayan pyramids, such as the Temple of Kukulcan in Chichén 
Itz4; the name Mayan diamond would have been more appropriate. 
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southeast side of the diamond, at the same height where it started. See Fig- 
ure 1.23 for an illustration. If we add i initial NE steps and i final SE steps to 
the (i+ 1)th path for each i, the result will be a routing of Schréder paths from 
A = {-(2n),...,—2,0} to B= {0,2,...,2n}. In fact this correspondence is a 
bijection [76]. 


We will prove in Section 1.4.2 that 


detH,(A) = 2")? 
Ns) foralln>O <=> Ais the Schréder sequence. 


det H/,(A) = 2"+)/2 


It will follow that 


number of domino tilings of the Aztec diamond AD, = 2""+))/?, 
This elegant result is originally due to Elkies, Kuperberg, Larsen, and Propp. 
For several other proofs, see [73, 74]. 


1.4.2 Computing determinants 


In light of Section 1.4.1, it is no surprise that combinatorialists have become talented 
at computing determinants. Fortunately, this is a very classical topic with connections 
to many branches of mathematics and physics, and by now there are numerous gen- 
eral techniques and guiding examples available to us. Krattenthaler’s surveys [122] 
and [123] are excellent references that have clearly influenced the exposition in this 
section. We now highlight some of the key tools and examples. 


1.4.2.1 Is it known? 


Of course, when we wish to evaluate a new determinant, one first step is to check 
whether it is a special case of some known determinantal evaluation. Starting with 
classical evaluations such as the Vandermonde determinant 


det(x!")i<ijcn= [] (xj-%), (1.10) 


l<i<j<n 
there is now a wide collection of powerful results at our disposal. A particularly 
useful one [122, Lemma 3] states that for any x1,...,%n,@2,.--,€n,02,...,D, we have: 
det [ (xi +b) Ese (x; +b) (x +aj+1) aes (x; +an)| ree 
= [] @-x) [] Ge). Gj) 


1<i<j<n 1<i<j<n 


For instance, as pointed out in [89] and [122], the Catalan determinant (1.9) is a 
special case of this formula. Recognizing it as such is not immediate, but the product 
formula for Catalan numbers gives an indication of why this is feasible. 
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In fact, here is a counterintuitive principle: Often the easiest way to prove a deter- 
minantal identity is to generalize it. It is very useful to introduce as many parameters 
as possible into a determinant, while making sure that the more general determi- 
nant still evaluates nicely. We will see this principle in action several times in what 
follows. 


1.4.2.2, Row and column operations 


A second step is to check whether the standard methods of computing determinants 
are useful: Laplace expansion by minors, or performing row and column operations 
until we get a matrix whose determinant we can compute easily. For example, recall 
the determinant Lo(K,,) of the (1 — 1) x (n—1) reduced Laplacian of the complete 
graph K,,, discussed in Section 1.4.1.3. We can compute it by first adding all rows to 
the first row, and then adding the first row to all rows: 


n—-1 —l tee —1 1 1 ee 1 
—l on —1 —-l n-l1 ::: —l1 
det Lo (K;,) = F . r : = 
—l —l n—-1 —l —l n—-1 
1 1 1 
a On ee. 
0 O «=. n 


reproving Theorem 1.4.6.1. 


1.4.2.3 Identifying linear factors 


Many nxn determinants of interest have formulas of the form detM(x) = 
cL;(x)---L,(x) where c is a constant and the L;(x) are linear functions in the vari- 
ables x = (x1,...,x,). We may prove such a formula by first checking that each L;(x) 
is indeed a factor of M, and then computing the constant c. 

The best known application of this technique is the proof of the formula (1.10) 
for Vandermonde’s determinant V(x1,...,X,). If x; =x ; for i ~ j, then rows i and 
j are equal, and the determinant is 0. It follows that x; — x; must be a factor of the 


polynomial det V (x1,...,x,). Since this polynomial is homogeneous of degree Gy it 


n—1 


must equal a constant times |]; ;(x; —x;). Comparing the coefficients of cape ae, 


we see that the constant equals 1. 

A similar argument may be used to prove the more general formula (1.11). 

To use this technique, it is sometimes necessary to introduce new variables into 
our determinant. For example, the formula det(i/~!),<; j<, = 112" ?---(n—1)! 
cannot immediately be treated with this technique. However, the factorization of the 
answer suggests that this may be a special case of a more general result where this 
method does apply; in this case, Vandermonde’s determinant. 
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1.4.2.4 Computing the eigenvalues 


Sometimes we can compute explicitly the eigenvalues of our matrix, and multiply 
them to get the determinant. One common technique is to produce a complete set of 
eigenvectors. 


1. (The Laplacian of the complete graph K,,) Revisiting the example above, the 
Laplacian of the complete graph is L(K,,) =nI—J where / is the identity matrix 
and J is the matrix all of whose entries equal 1. We first find the eigenvalues 
of J: 0 is an eigenvalue of multiplicity n — 1, as evidenced by the linearly inde- 
pendent eigenvectors e; —e2,...,€,—_1 — €,. Since the sum of the eigenvalues is 
tr(J) =n, the last eigenvalue is n; an eigenvector is e; +---+e,_1. Now, if v is 
an eigenvector for J with eigenvalue A, then it is an eigenvector for nJ — J with 
eigenvalue n— A. Therefore the eigenvalues of nJ — J are n,n,...,n,0. Using 
Theorem 1.4.5, we have reproved yet again that detLo(Kn) = +(n"~!) =n™?. 


2. (The Laplacian of the n-cube C,) A more interesting example is the re- 
duced Laplacian Lo(C,) of the graph of the n-dimensional cube, from The- 
orem 1.4.6.4. By producing explicit eigenvectors, one may prove that if the 
Laplacians L(G) and L(H) have eigenvalues {Aj : 1 <i<a} and {uj :i< 
j <b} then the Laplacian of the product graph L(G x H) has eigenvalues 
{Ait pj: 1<i<a,1 <j <b}. Since C; has eigenvalues 0 and 2, this im- 
plies that C, = C) x --- x C) has eigenvalues 0,2,4,...,2 with multiplicities 

aE Cece: respectively. Therefore the number of spanning trees of the 


o) > \a)>: 
cube C,, is 


detLo(Cy) = 5 2al (Qn) ln) = 22"-9-1) (23) ...n Ge), 


3. (The perfect matchings of a rectangle) An even more interesting example 
comes from the perfect matchings of the a x b rectangle, which we discussed 
in Section 1.4.1.5. Let V be the 4mn-dimensional vector space of functions 
f : [2m] x [2n] > C, and consider the linear transformation L: V > V given by 


where f(x,y) =0 when x € {0,a+1} ory € {0,b+ 1}. The matrix of this lin- 
ear transformation is precisely the one we are interested in. A straightforward 
computation shows that the following are eigenfunctions and eigenvalues of L: 


ku. Int ku It 
gk4(%,y) = sin sin Xi = 2.cos 4 +2icos 


for 1 <k <aand1</ <b. (Note that g, (x,y) =0 forx € {0,a} ory € {0,b}.) 
This is then the complete list of eigenvalues for L, so 


kt It 
detL = 2” 
e TIT (coz 100s), 


1/=1 


which is easily seen to equal the expression in Theorem 1.4.13. 
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1.4.2.5 LU factorizations 


A classic result in linear algebra states that, under mild hypotheses, a square matrix 
M has a unique factorization 
M=LU 


where L is a lower triangular matrix and U is an upper triangular matrix with all diag- 
onal entries equal to 1. Computer algebra systems can compute the LU-factorization 
of a matrix, and if we can guess and prove such a factorization it will follow imme- 
diately that detM equals the product of the diagonal entries of L. 

An interesting application of this technique is the determinant 


det(ged(i, j))1<i,i<n =[ J OC), (1.12) 
i=1 


where p(k) = {iE N : (ged(i,k) = 1 and 1 <i< k} is Euler’s totient function. This 
is a special case of a more general formula for semilattices that is easier to prove. 
For this brief computation, we assume familiarity with the Mobius function uw and 
the zeta function € of a poset; these will be treated in detail in Section 1.5.5.3. 

Let P be a finite meet semilattice and consider any function F : P x P > k. We 
will prove the Lindstro6m—Wilf determinantal formula: 


saF eVarlnne= TI (Emowre Pr) i) (1.13) 


peEP \r=p 


Computing some examples will suggest that the LU factorization of F is F = MZ 
where 


M, pq — , , 
otherwise, 0 otherwise. 


ioe ifp <q, oe f if p> 4q, 
0 pq — 
This guess is easy to prove, and it immediately implies (1.13). In turn, applying the 
Lindstrém—Wilf to the poset of integers {1,...,} ordered by reverse divisibility and 
the function F (x,y) =x, we obtain (1.12). 

Another interesting special case is the determinant 


det(x™k0PV9)) , cep = I] (ane Xp} (1/x)) ; 


peP 


where Y/,, mes x) is the characteristic polynomial of the interval [x, i]. When P is the 
partition lattice II,,, this determinant arises in Tutte’s work on the Birkhoff-Lewis 
equations [203]. 


1.4.2.6 Hankel determinants and continued fractions 


For Hankel determinants, the following connection with continued fractions [209] is 
extremely useful. If the expansion of the generating function for a sequence fo, f{,... 
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as a J-fraction is 


then the Hankel determinants of fo, f,... equal 
det Hn (A) = fbr be be adn 


For instance, using the generating function for the Schroder numbers r,, it is easy to 
prove that 


nL ui 
ee se tp = 
n=0 1—2x 1 


Therefore 
detH,(R)=2""-D/2, det (R) = 2"™F0)/2, 


as stated in Example 5 of Section 1.4.1.6. 

By computer calculation, it is often easy to guess J-fractions experimentally. With 
a good guess in place, there is an established procedure for proving their correctness, 
rooted in the theory of orthogonal polynomials; see [122, Section 2.7]. 


Dodgson condensation. It is often repeated that Lewis Carroll, author of Alice in 
Wonderland, was also an Anglican deacon and a mathematician, publishing under his 
real name, Rev. Charles L. Dodgson. His contributions to mathematics are discussed 
less often, and one of them is an elegant method for computing determinants. 

To compute an n x n determinant A, we create a square pyramid of numbers, 
consisting of n+ 1 levels of size n+ 1,n...,1, respectively. On the bottom level we 
place an (n+ 1) x (n +1) array of 1s, and on the next level we place the n x n matrix 
A. Each subsequent floor is obtained from the previous two by the following rule: 
Each new entry is given by f = (ad — bc)/e where f is directly above the entries 

a b 
d 
determinant. For example, the computation 


and two floors above the entry e. * The top entry of the pyramid is the 


NNrAMN 


1 
1 
1 
1 
1 


SS Se Se 
— epee 
SS ele 


1 
1 
l1|—-> 
1 
1 


shows that the determinant of the 4 x 4 determinant is 21. 
Dodgson’s condensation method relies on the following fact, due to Jacobi. If A 


is ann Xn matrix and Aj, i, ; j,,...,j, denotes the matrix A with rows ij,...,i, and 
columns j;,..., 4 removed, then 
detA -detAj n-1n = detA,.; -detAy., —detA1;,-detAn,1. (1.14) 


*Special care is required when Os appear in the interior of the pyramid. 
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This proves that the numbers appearing in the pyramid are precisely the determinants 
of the “contiguous” submatrices of A, consisting of consecutive rows and columns. 
If we have a guess for the determinant of A, as well as the determinants of its 
contiguous submatrices, Dodgson condensation is an extremely efficient method to 
prove it. All we need to do is to verify that our guess satisfies (1.14). 
To see how this works in an example, let us use Dodgson condensation to prove 
the formula in Section 1.4.1.6 for R,, = det G os) peetes the number of stacks of 
<i,j<n 
unit cubes in the corner of ann x n xn box. The first step is to guess the determinant 
of the matrix in question, as well as all its contiguous submatrices; they are all of the 
form R(a,b,c) = det ( ae ) ee where a+b = 2n. This more general determinant 
is equally interesting combinatorially: it counts the stacks of unit cubes in the corner 
of an a x b x c box. By computer experimentation, it is not too difficult to arrive at 


the following guess: 


a+b ie ape ay 3 Oy es | 
Rabe) =aet| ( )| [1—— >. 
I<i,jSc tk—2 


a+i-—j i=l j=lk=1 TJ 4 


Proving this formula by Dodgson condensation is then straightforward; we just need 
to check that our conjectural product formula holds for c = 0,1 and that it satisfies 
(1.14); that is, 


R(a,b,c + 1)R(a,b,c— 1) = R(a,b,c)? — R(a+1,b—1,c)R(a—1,b+1,c). 


For more applications of Dodgson condensation, see for example [3]. 

There is a wonderful connection between Dodgson condensation, Aztec dia- 
monds, and alternating sign matrices, which we now describe. Let us construct 
a square pyramid of numbers where levels n +2 and n+ | are given by two matrices 
Y = (vis)i<ij<nt2 and x = (xi) 1<i,j<n41, respectively, and levels n—1,...,2,1 are 
computed in terms of the lower rows using Dodgson’s recurrence f = (ad — bc) /e. 
Let f(x,y) be the entry at the top of the pyramid. 

Remarkably, all the entries of the resulting pyramid will be Laurent monomials 
in the x;;8 and y;;s; that is, their denominators are always monomials. This is obvious 
for the first few levels, but it becomes more and more surprising as we divide by 
more and more intricate expressions. 

The combinatorial explanation for this fact is that each entry in the (n — k)th level 
of the pyramid encodes the domino tilings of an Aztec diamond AD,. For instance, 
if n = 3, the entry at the top of the pyramid is 


— ¥11%22N33,— X11X23X32_——- X12X21X33.  X12X21%423132 
i (x, y) ~~ + 
22933 22933 Y22Y33 X22Y22Y33 
X12%21X23432 | X12%23431 4 X13X21X32 -—- X13.X31X32 
T 
X22V2332 2332 23932 23932 


There is a simple bijection between the eight terms of f3 and the eight domino 
tilings of AD. Regard a tiling of AD2 as a graph with vertices on the underlying 
lattice, and add a vertical edge above and below the tiling, and a horizontal edge to the 
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a g 9 1 9g 
O 1 
e e 

O O q 

——P —P- e e . 
a 4g 

1 9 9g 


Figure 1.24 
A domino tiling of AD» and the corresponding monomial in f2(x,y). 


left and to the right of T. Now rotate the tiling 45°. Record the degree of each vertex, 
ignoring the outside corners on the boundary of the diamond, and subtract 3 from 
each vertex. This leaves us with an n x n grid of integers within an (n+ 1) x (n+ 1) 
grid of integers. Assign to it the monomial whose x exponents are given by the outer 
grid and whose y exponents are given by the inner grid. For example, the tiling in 
Figure 1.24 corresponds to the monomial (x12%23%31) /(v23y32). 

In general, this gives a bijection between the terms of f,(x,y) and the domino 
tilings of the Aztec diamond AD, One may also check that there are no cancella- 
tions, so Dodgson condensation tells us that the number m,, of terms in f;, satisfies 
Mn—{My+1 = 2m. This gives an alternative proof that the Aztec diamond AD,, has 
2"("+1)/2 domino tilings. 

We may also consider the patterns formed by the x;;s by themselves (or of the 
yijS by themselves). In each individual monomial of f,(x,y), the exponents of the 
x;j8 form an n X n alternating sign matrix (ASM): a matrix of 1s, Os, and —1s such 
that the nonzero entries in any row or column alternate 1,—1,...,—1,1. Similarly, 
the negatives of the exponents of the y;;s form an ASM of size n — 1. 

Alternating sign matrices are fascinating objects in their own right, with connec- 
tions to representation theory, statistical mechanics, and other fields. The number of 
alternating sign matrices of size n is 


1!4!7!---(3n—2)! 
ni (n+1)!(n+2)!---(Qn—1)) 


For details on the history and solution of this difficult enumeration problem see [44, 
167, 222]. 


1.5 Posets 


This section is devoted to the enumerative aspects of the theory of partially ordered 
sets (posets). Section 1.5.1 introduces key definitions and examples. Section 1.5.2 
discusses some families of lattices that are of special importance. In Section 1.5.3 we 
count chains and linear extensions of posets. 
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The remaining sections are centered around the Mébius Inversion Formula, 
which is perhaps the most useful enumerative tool in the theory of posets. This for- 
mula helps us count sets that have an underlying poset structure; it applies to many 
combinatorial settings of interest. 

In Section 1.5.4 we discuss the Inclusion-Exclusion Formula, a special case of 
great importance. In Section 1.5.5 we introduce Mobius functions and the Mobius 
Inversion Formula. In particular, we catalog the Mobius functions of many important 
posets. The incidence algebra, a nice algebraic framework for understanding and 
working with the Mobius function, is discussed in Section 1.5.5.3. In Section 1.5.5.4 
we discuss methods for computing Mobius functions of posets, and sketch proofs for 
the posets of Section 1.5.5. Finally, in Section 1.5.6, we discuss Eulerian posets and 
the enumeration of their flags, which gives rise to the ab-index and cd-index. 


1.5.1 Basic definitions and examples 


A partially ordered set or poset (P, <) is a set P together with a binary relation <, 
called a partial order, such that 


e Forall p € P, we have p < p. 
e Forallp,géP,ifp<qandq<pthenp=gq. 
e Forallp,g,r¢P,ifp<qandg <r then p <r. 


We say that p <q if p< qand p #q. We say that p and q are comparable if p < q 
or p > q, and they are incomparable otherwise. We say that g covers p if g > p and 
there is no r € P such that g > r > p. When q covers p we write q > p. 


Example 1.5.1 Many sets in combinatorics come with a natural partial order, and 
often the resulting poset structure is very useful for enumerative purposes. Some of 
the most important examples are the following: 


1. (Chain) The poset n = {1,2,...,n} with the usual total order. (n> 1) 
2. (Boolean lattice) The poset 24 of subsets of a set A, where S < T if SCT. 


3. (Divisor lattice) The poset D, of divisors of n, where c < d if c divides d. 
(n> 1) 


4. (Young’s lattice) The poset Y of integer partitions, where A < w if Ay < yy; for 
all i. 


5. (Partition lattice) The poset I, of set partitions of [n], where 1 < p if 7 refines 
p; that is, if every block of p is a union of blocks of m. (n = 1) 


6. (Non-crossing partition lattice) The subposet NC, of 1, consisting of the non- 
crossing set partitions of |n], where there are no elements a <b<c<d such 
that a,c are together in one block and b,d are together in a different block. 
(n> 1) 
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7. (Bruhat order on permutations) The poset S, of permutations of |n|, where 1 
covers p— if 1 is obtained from p by choosing two adjacent numbers pj =a < 
b = pi+1 in p and exchanging their positions. (n > 1) 


&. (Subspace lattice) The poset L(F%) of subspaces of a finite dimensional vector 
space F%, where U < V if U is a subspace of V. (n = 1, q a prime power) 


9. (Distributive lattice) The poset J(P) of order ideals of a poset P (subsets I C P 
such that j € P andi < j imply i € P) ordered by containment. 


10. (Face poset of a polytope) The poset F (P) of faces of a polytope P, ordered by 
inclusion. 


11, (Face poset of a subdivision of a polytope) The poset a of faces of a subdi- 
vision J of a polytope P ordered by inclusion, with an additional maximum 
element. 


12, (Subgroup lattice of a group) The poset L(G) of subgroups of a group G, or- 
dered by containment. 


The Hasse diagram of a finite poset P is obtained by drawing a dot for each 
element of P and an edge going down from p to g if p covers q. Figure 1.25 shows 
the Hasse diagrams of some of the posets above. In particular, the Hasse diagram of 
2"! is the 1-skeleton of the n-dimensional cube. 


123 321 
4 123 
12 23 312 231 
9 
2 12/3 128 
1 3 132 213 
1 
@ 23 


12/3 1 
Figure 1.25 

The Hasse diagrams of the chain 4, Boolean lattice 2 
lattice IT;, and Bruhat order $3. 


ie) 


in) 


B] divisor lattice Dg, partition 


A subset Q of P is a chain if every pair of elements is comparable, and it is 
an antichain if every pair of elements is incomparable. The length of a chain C is 
|C| — 1. If there is a rank function 7 : P > N such that r(x) = 0 for any minimal 
element x and r(y) = r(x) + 1 whenever y > x, then P is called graded or ranked. 
The largest rank is called the rank or height of P. The rank-generating function of 
a finite graded poset is 

R(P3x) = 3 xP), 


peP 


All the posets of Example 1.5.1 are graded except for subgroup lattices. 
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A poset P induces a poset structure on any subset Q C P; a special case of interest 
is the interval [p,g] = {r © P : p<r<q}. We call a poset locally finite if all its 
intervals are finite. Given posets P and Q on disjoint sets, the direct sum P+ Q is 
the poset on PUQ inheriting the order relations from P and Q, and containing no 
additional order relations between elements of P and Q. The direct product P x Q 
is the poset on P x Q where (p,q) < (p',q') if pp < p' andg <q’. 

We have already seen example: of product posets. The Boolean lattice is 24 & 
2x--+x 2. Also, ifn = pil - pik is the prime factorization of n, then D, © (t; +1) x 

x (ti + 1). 


1.5.2 Lattices 


A poset is a lattice if every two elements p and q have a least upper bound p V g and 
a greatest lower bound p /q, called their meet and join, respectively. We will see 
this additional algebraic structure can be quite beneficial for enumerative purposes. 


Example 1.5.2 All the posets in Example 1.5.1 are lattices, except for the Bruhat 
order. In most cases, the meet and join have easy descriptions. In n, the meet and 
join are the minimum and maximum, respectively. In 24 they are the intersection and 
union. In Dy, they are the greatest common divisor and least common multiple. In Y 
they are the componentwise minimum and maximum. In YI, and in NC, the meet of 
two partitions 1 and p is the collection of intersections of a block of m and a block 
of p. In L(F 4) the meet and join are the intersection and the span. In J(P) they are 
the intersection and the union. In F (P) the meet is the intersection. In L(G) the meet 
is the intersection. 


Any lattice must have a unique minimum element 0 and maximum element 1. An 
element covering 0 is called an atom; an element covered by Tis called a 1 coatom. 
To prove that a finite poset P is a lattice, it is sufficient to check that it has a Tand that 
any x,y € P have a meet; then the join of x and y will be the (necessarily non-empty) 
meet of their common upper bounds. Similarly, it suffices to check that P has a 0 and 
that any x,y € P have a join. 


Distributive lattices. A lattice L is distributive if the join and meet operations 
satisfy the distributive properties: 


V(yAz) = (xVy)A(xV2), xA(yVz) = (xAy) V (xAz) (1.15) 


for all x,y,z € L. To prove that L is distributive, it is sufficient to verify that one of 
the equations in (1.15) holds for all x,y,z € L. 


Example 1.5.3 There are several distributive lattices in Example 1.5.1: the chains 
n, the Boolean lattices 24, the divisor lattices Dy, and Young’s lattice Y. This follows 
from the fact that the pairs of operations (min,max), (gcd,lem) and (N,V) satisfy 
the distributive laws. The others are not necessarily distributive; for example, 113 


and §3. 
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The most important, and in fact, the only, source of finite distributive lattices is 
the construction of Example 1.5.1.9: Given a poset P, a downset or order ideal / is 
a subset of P such that if i € J and j <ithen j € J. A principal order ideal is one of 
the form P<p ={q € P: q< p}. Let J(P) be the poset of order ideals of P, ordered 
by inclusion. 


Theorem 1.5.4 (Fundamental Theorem for Finite Distributive Lattices) A poset 
Lis a distributive lattice if and only if there exists a poset P such that L= J(P). 


Sketch of Proof. Since the collection of order ideals of a poset P is closed under 
union and intersection, J(P) is a sublattice of 2”. The distributivity of 2” then implies 
that J(P) is a distributive lattice. 

For the converse, let L be a distributive lattice, and let P be the set of join- 
irreducible elements of L; that is, the elements p > 0 that cannot be written as 
p=4qVr for q,r < p. These are precisely the elements of L that cover exactly one 
element. The set P inherits a partial order from L, and this is the poset such that 
L&J(P). The isomorphism is given by 


g:J(P) —> L 


Ie Vp 
pel 
and the inverse map is given by @-!(/) ={peP: p<ll}. a 


NI 


Figure 1.26 
A poset and the corresponding distributive lattice. 


Theorem 1.5.4 extends to some infinite posets with minor modifications. Let 
J;(P) be the set of finite order ideals of a poset P. Then the map P ++ J;(P) is a 
bijection between the posets whose principal order ideals are finite and the locally 
finite distributive lattices with 0. 
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Example 1.5.5 The posets P of join-irreducibles of the distributive lattices L™ J(P) 
of Example 1.5.3 are as follows. For L=n, P =n—1 is a chain. For L= 24, P= 
1+---+1 is an antichain. For L = Dy, where n =p: pi P=t)+---+t is the 
disjoint sum of k chains. For L=Y, P =N x Nis a “quadrant.” 


Theorem 1.5.4 explains the abundance of cubes in the Hasse diagram of a dis- 
tributive lattice L. For any element / € L covered by n elements /|,...,/, of L, the 
joins of the 2” subsets of {1),...,/,} are distinct, and form a copy of the Boolean 
lattice 2!" inside L. The dual result holds as well. 

The width of a poset P is the size of the largest antichain of P. Dilworth’s the- 
orem [68] states that this is the smallest integer w such that P can be written as the 
disjoint union of w chains. 


Theorem 1.5.6 The distributive lattice J(P) can be embedded as an induced sub- 
poset of the poset N”, where w is the width of P. 


Proof. Decompose P as the disjoint union of w chains C),...,C,. The map 
o:J(P) — NY” 
Toe (INC I,..., ZAC) 
gives the desired inclusion. | 


Geometric lattices. Now we introduce another family of lattices of great impor- 
tance in combinatorics. We say that a lattice L is: 


e semimodular if the following two equivalent conditions hold: 


-— Lis graded and r(p)+r(q) >r(pAq)+r(pVq) forall p,g € L. 
— Ifpandq both cover p/q, then pV q covers both p and gq. 


e atomic if every element is a join of atoms. 


e geometric if it is semimodular and atomic. 


Example 1.5.7 In Figure 1.25, the posets 21) and Ty are geometric, while the posets 
4, Dig, and 83 are not. 


Not surprisingly, the prototypical example of a geometric lattice comes from a 
natural geometric construction, illustrated in Figure 1.27. Let A = {v,...,vn} be a 
set of vectors in a vector space V. A flat is a subspace of V generated by a subset of 
A. We identify a flat with the set of vjs that it contains. Let L, be the set of flats of A, 
ordered by inclusion. Then Ly, is a geometric lattice. 

The theory of geometric lattices is equivalent to the rich theory of matroids, 
which is the subject of Section 1.8. 
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Figure 1.27 
A vector configuration t,u, v, w (where t,u, v are coplanar in R*) and the correspond- 
ing geometric lattice. 


Supersolvable lattices. A lattice L is supersolvable if there exists a maximal 
chain, called an M-chain, such that the sublattice generated by C and any other chain 
of L is distributive. [182] 

Again unsurprisingly, an important example comes from supersolvable groups, 
but there are several other interesting examples. Here is a list of supersolvable lat- 
tices, and an M-chain in each case. 


1. Distributive lattices: every maximal chain is an M-chain. 

2. Partition lattice I1,: 1|2|--+|2 < 12|3|---|n < 123|4]---|n<--- << 123---n. 
3. Noncrossing partition lattice NC;,,: the same chain as above. 
4 


. Lattice of subspaces L(IF7) of the vector space Fj over a finite field F,: every 
maximal chain is an M-chain. 


5. Subgroup lattices of finite supersolvable groups G: an M-chain is given by any 
normal series 1 = Hp <H) <---<H, = G where each H; is normal and H;/H;_1 
is cyclic of prime order. 


Fortunately, there is a simple criterion to verify semimodularity. An R-labeling 
of a poset P is a labeling of the edges of the Hasse diagram of P with integers such 
that for any s < ¢ there exists a unique maximal chain C from s to f. 


Theorem 1.5.8 [139] A finite graded lattice of rank n is supersolvable if and only if 
it has an R-labeling for which the labels on every maximal chain are a permutation 


of {1,...,n}. 


1.5.3 Zeta polynomials and order polynomials 


The zeta polynomial of a finite poset P counts the multichains of various lengths 
in P. A multichain of length & in P is a sequence of possibly repeated elements 
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to,t1,-.-,t, € P such that to <t) <--- <t,. Let 
Zp(k) = number of multichains of length k — 2 in P (k > 2). (1.16) 


There is a unique polynomial Zp(k) satisfying (1.16) for all integers k > 2; it is given 
by 


k—-2 
Zn(k) =Dn(4 5): (1.17) 


i>2 
where b; is the number of chains of length i—2 in P. This polynomial is called the 
zeta polynomial of P. 


Example 1.5.9 The following posets have particularly nice zeta polynomials: 


Il. P=n: 


2. P=By: 


3. P=WNC,: (Kreweras, [124]) 


2)= 7 (40) 


The order polynomial of P counts the order-preserving labelings of P; it is de- 
fined by 


Qp(k) = number of maps f : P — [k] such that p < g implies f(p) < f(q) 


for k € N. The next proposition shows that, once again, there is a unique polynomial 
taking these values at the natural numbers. 


Proposition 1.5.10 For any poset P, Qp(k) = Zjp)(k). 


Proof. An order-preserving map f : P — [k] gives rise to a sequence of order ideals 
fo'AY © f71(41,2}) Ce fT (41,...,4}), which is a multichain in J(P). Con- 
versely, every sequence arises uniquely in this way. | 


A linear extension of P is an order-preserving labeling of the elements of P with 
the labels 1,..., = |P|, which extends the order of P; that is, a bijection f : P > |n] 
such that p <q implies f(p) < f(q). Let 


e(P) = number of linear extensions of P. 


It follows from Proposition 1.5.10 and (1.17) that the order polynomial Qp has degree 
|P|, and leading coefficient e(P)/|P|!. 
The following is a method for computing e(P) recursively. 
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Proposition 1.5.11 Define e: J(P) > N recursively by 


1 if 1 =0, 
e(I) = yi e(J) otherwise. 
J<l 


Then e(1) is the number e(P) of linear extensions of P. 


Proof. Let p1,..., px be the maximal elements of P. In a linear extension of P, one 
of p1,..-, px has to be labeled n, and therefore 


e(P) =e(P\{pi}) +--+ +e(P\ {px })- 


This is equivalent to the desired recurrence. | 


It is useful to keep in mind that J(P) is a subposet of NY for w = w(P). The 
recurrence of Proposition 1.5.11 generalizes Pascal’s triangle, which corresponds to 
the case P = N+N. When we apply it to P= N+---+N, we get the recursive 
formula for multinomial coefficients. 


Example 1.5.12 In some special cases, the problem of enumerating linear exten- 
sions is of fundamental importance. 


@e(m +--+ my) = ("A 


e e(2xn)=C, = ,4(7). 


e Let T be a tree poset of n elements, such that the Hasse diagram is a tree 


rooted at 0. For each vertex v let t, = |T>,| = |{w € T : w> v}|. Then 


n! 
Iherty 


e(T) 


e Let A be a Ferrers diagram of n cells, partially ordered by decreeing that 
each cell is covered by the cell directly below and the cell directly to the 
right, if they are in A. The hook H, of a cell c consists of cells on the same 
row and to the right of c, those on the same column and below c, and c itself. 
Let h, = |H,|. Then 

n! 
Teen he 
This is the dimension of the irreducible representation of the symmetric group 
S, corresponding to A. [173] 


e(A) 


1.5.4 The inclusion-exclusion formula 


Our next goal is to discuss one of the most useful enumerative tools for posets: 
Mobius functions and the M6bius inversion theorem. Before we do that, we de- 
vote this section to a special case that preceded and motivated them: the inclusion- 
exclusion formula. 
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Theorem 1.5.13 (Inclusion-Exclusion Formula) For any finite sets A,,...,An CX, 
we have 


I. |A,U++-UAn| =) Ai|— Yo AINA; + YY AINA ;NAg|—++-E|ALN---MAal. 
i i<j i<j<k 
2. |AiN-+-NAn| = |X|— EIA] + ¥ |AINA;|—-+- EAL N+ Ay. 
i i<j 


Proof. — It suffices to prove one of these two equivalent equations. To prove the first 
one, consider an element x appearing in k > 1 of the given sets. The number of times 
that x is counted in the right-hand side is k — (5) tet (i) =1. a 


We now present a slightly more general formulation. 


Theorem 1.5.14 (Inclusion-Exclusion Formula) Let A be a set and consider two 
functions f_, fs 124 —+k from 24 to a field k. Then 


1. fe(8)= Y fL(L) forSCA => f-(S)= ¥ (-1) SIAL) for SCA. 
TOS 


TDS 


2. fe(S)= YP f(T) forSCA = f-(S)= Y (“1 fe(7) for SCA. 
TCS TCS 

The most common interpretation is the following. Suppose we have a set U of 
objects and a set A of properties that each object in U may or may not satisfy. If we 
know, for each S C A, the number fS(S) of elements having at /east the properties 
in S (or the number f<(S) of elements having at most the properties in S), then we 
obtain, for each S C A, the number f_(S) of elements having exactly the properties in 
S. We are often interested in the number f—(@) or f—(A) of elements satisfying none 
or all of the given properties. 

Theorem 1.5.14 has a simple linear algebraic interpretation. Consider the two 
24 x 24 matrices C ,D whose non-zero entries are Cs 7 = 1 for $C T, and Ds 7 = 
(—1)!7-SI for S C T. Then the inclusion-exclusion formula is equivalent to the as- 
sertion that C and D are inverse matrices. This can be proved directly, but we prefer 
to deduce it as a special case of the M6bius inversion formula (Theorem 1.5.16). We 
now present two applications. 


Derangements. One of the classic applications of the inclusion-exclusion formula 
is the enumeration of the derangements of [n], which are the permutations z € S, 
such that 2(i) Ai for all i. Let A = {A,...,An} where A; is the property that a(i) =i. 
Then f(T) = (n—|T])!, so the number D,, of derangements of [n] is 


Dy. = f-(0) =E-1)" Fe) = ¥ (f(D 


k=0 


2. fi Ted 1 
NG at on, 


It follows that D,, is the integer closest to n!/e. 
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Discrete derivatives. Consider the k-vector space I of functions f : Z — k. The 
discrete derivative of f is the function Af given by Af(n) = f(n+1)— f(n). We 
now wish to show that, just as with ordinary derivatives, 


Aer f =0if and only if f is a polynomial of degree at most d. 


This was part of Theorem 1.3.6. Regarding A as a linear operator on I’, we have 
A =E-—1 where Ef(n) = f(n+1) and 1 is the identity. Then A‘ = (E —1)k = 
yk 9 (‘)E'(-1)*-4, so the kth discrete derivative is A‘ f(n) =D* 9(—1)*(4) f(nti). 

The functions f<(S) = f(n+|S|) and f-(S) = A* f(|S|) satisfy Theorem 1.5.14.2, 
so we have f(n +k) = Yo (‘)A‘f(n). (This is equivalent to EX = (A+1)*.) If 
A‘+! f = 0, this gives f(k) = V4 (‘al f (0), which is a polynomial in k of degree 
at most d. The converse follows from the observation that A lowers the degree of a 
polynomial by 1. 


1.5.5 Mobius functions and Mobius inversion 
1.5.5.1 The Mobius function 


Given a locally finite poset P, let Int(P) = {|x,y] : x,y © P,x < y} be the set of 
intervals of P. The (two-variable) Mobius function of a poset P is the function y : 
Int(P) > Z defined by 


y ml ={) ee (1.18) 


perxq 0 otherwise. 


Here we are denoting U(p,q) = LU ([p,q]). We will later see that the Mébius function 
can be defined equivalently by the equations: 


y utead={) amis (1.19) 


pera otherwise. 


When P has a minimum element 0, the (one-variable) Mébius function yi : P > 
Z is U(x) = u(0,x). If P also has a 1, the Mébius number of P is L(P) = (0,1). 

Computing the Mobius function is a very important problem, because the Mobius 
function is the poset analog of a derivative; and as such, it is a fundamental invariant 
of a poset. This problem often leads to very interesting enumerative combinatorics, 
as can be gleaned from the following gallery of Mobius functions. 


Theorem 1.5.15 The Mobius functions of some key posets are as follows. 


I, (Chain) P =n: 
1 ifj=i 
Mali, j)=§—-1 if j=i+l 
0) otherwise. 
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. (Boolean lattice) P = 24: 


bya (8,T) = (-1)P-, 


. (Divisor lattice) P = Dy: We have Lp, (k,l) = w(L/k) where 


(m) (-1)' ifmis a product of t distinct primes, and 
m) = 
F 0 otherwise. 


is the classical Mébius function from number theory. 


. (Young’s lattice) P =Y: 


(—1)l#-41 if —A has no two adjacent squares, and 


0 otherwise. 


. (Partition lattice) P =T1,: The Mobius number of 11, is 


w(Mn) = (—1)"""(n— 1}, 


from which a (less elegant) formula for the complete Mébius function can be 
derived. 


. (Non-crossing partition lattice) P = NC,: The Mobius number of NC, is 


U(NCn) = (—1)""!Ch-1, 


where C;,_ is the (n—1)th Catalan number. This gives a (less elegant) formula 
for the complete Mobius function. 


(Bruhat order) P = Sp: 
(u,v) = (-1)-, 


where the length €(w) of a permutation w € S, is the number of inversions 
(i, j) where 1 <i < j <nand w; > wj. (There is a generalization of this result 
to the Bruhat order on any Coxeter group W, or even on a parabolic subgroup 
W/; see Section 1.5.5.4.) 


. (Subspace lattice) P = L(F%): 


u(U,V) = (-1)4q@), 


where d = dimV — dimU. 


. (Distributive lattice) L = J(P): 


(I,J) = (—1)¥~!|_ if J—Lis an antichain in P, and 
es 0 otherwise. 
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10. (Face poset of a polytope) L = F (P): 
UF. G) = (aya 


11. (Face poset of a subdivision J of a polytope P) L= T: 


(ieee if G <1 
BPG) =< (—1)aeP dink! GS 1 and F is not on the boundary of P 
0 ifG= 1 and F is on the boundary of P. 


12. (Subgroup lattice of a finite p-group) If |G| = p" for p prime, n © N, then in 
L=L(G): 


u(A,B) = ae if A is anormal subgroup of B and B/A = Z*, and 
esas, |@ 


otherwise. 


Some of the formulas above follow easily from the definitions, while others re- 
quire more sophisticated methods. In the following sections we will develop some 
of the basic theory of Mobius functions and discuss the most common methods for 
computing them. Along the way, we will sketch proofs of all the formulas above. 

It is worth remarking that a version of item 11 of Theorem 1.5.15 holds more 
generally for the face poset of any finite regular cell complexes I’ such that the un- 
derlying space |I'| is a manifold with or without boundary; see [194, Prop. 3.8.9] for 
details. 


1.5.5.2 Mobius inversion 


In enumerative combinatorics, there are many situations where we have a set U of 
objects, and a natural way of assigning to each object u of U an element f(u) of 
a poset P. We are interested in counting the objects in U that map to a particular 
element p € P. Often we find that it is much easier to count the objects in s that map 
to an element less than or equal to * p in P. The following theorem tells us that 
this easier enumeration is sufficient for our purposes, as long as we can compute the 
Mobius function of P. 


Theorem 1.5.16 (Mébius Inversion formula) Let P be a poset and let f,g: Pk 
be functions from P to a field kk. Then 


1. VpeEPg(p)=\V fq) <= YpeP f(p)= Y ulp.a)s(q) and 


q=p q=p 
2. VpEPg(p)=) fq) — YpeP f(p)=Y ula.p)s(q). 
qsp asp 


In his paper [169], which pioneered the use of the MO6bius inversion formula as 
a tool for counting in combinatorics, Rota described this enumerative philosophy as 
follows: 


*or greater than or equal to 
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It often happens that a set of objects to be counted possesses a natural 
ordering, in general only a partial order. It may be unnatural to fit the 
enumeration of such a set into a linear order such as the integers: instead, 
it turns out in a great many cases, that a more effective technique is to 
work with the natural order of the set. One is led in this way to set up a 
“difference calculus” relative to an arbitrary partially ordered set. 


Indeed, one may think of the Mobius function as a poset-theoretic analog of the 


Fundamental Theorem of Calculus: g is analogous to the integral of f, as it stores the 
cumulative value of this function. Like the Fundamental Theorem of Calculus, the 
Mobius inversion formula tells us how to recover the function f from its cumulative 
values. 


It is not difficult to prove Theorem 1.5.16 directly, but we will soon discuss an 


algebraic framework that really explains it. In the meantime, we discuss two key 
applications. We will see several other applications later on. 


Example 1.5.17 Mobius inversion is particularly important for chains, Boolean and 
divisor lattices. 


I. For P=N, Theorem 1.5.16.1 is a simple but important result for partial sums: 


n 


g(n) =) f (i) forné€N  ifandonlyif f(n)=g(n)—g(n—1) forneN 


i=0 


where g(—1) =0. 


. For P = 24, Theorem 1.5.16.1 is the inclusion-exclusion formula: 


g(S)= )) f(T) forallS CA if and only if 
TOS 


f(S)= ¥ (DI? Se(7) for all S CA. 
TOS 


. For P = Dy, Theorem 1.5.16.2 is the Mobius inversion formula from number 


theory: 
g(m)= ) f(d) form|n  ifandonly if f(m)= Y) w(d/m)g(d) for min. 


m|d|n m|d|n 
A typical application is the computation of Euler’s totient function 
p(n) =l{w: 1 <u<n, ged(u.n) = 1}, 


or more generally f(m) = |{u: 1 <u<n, gced(u,n) = m}| for m|n. Here 
U = [n|, and we assign to each u € U the divisor gcd(n,u) of n. There are 
g(m) =n/m multiples of m in U, so 


on) = f(t) = Palas =n(1-L) (1-2). 


d\n 
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We will be interested in applying the Mobius inversion formula in many other 
contexts, and for that reason it is important that we gain a deeper understanding of 
Mobius functions, and develop techniques to compute them. This is one of the main 
goals of the following sections. 


1.5.5.3 The incidence algebra 


The Mobius function has a very natural algebraic interpretation, which we discuss 
in this section. Given a field k, recall that a k-algebra A is a vector space over k 
equipped with a bilinear product. 

The incidence algebra /(P) of a locally finite poset P is the k-algebra of func- 
tions f : Int(P) — k from the intervals of P to k, equipped with the convolution 
product f - g given by 

f-sipr= ¥ flp.asla.r)  forp<r. 
psqsr 

Alternatively, let Mat(P) be the set of P x P matrices A with entries in k whose 
only nonzero entries a), 4 0 occur in positions where p < q in P. There is no canoni- 
cal way of listing the rows and columns of the matrix P ina linear order. We normally 
list them in the order given by a linear extension of P, so that the matrices in Mat(P) 
will be upper triangular. Then Mat(P) is a k-algebra under matrix multiplication, and 
it is clear from the definitions that 


1(P) = Mat(P). 


The unit and inverses. The product in /(P) is clearly associative, and has a unit 


1 ifp=q, 
1(p,q) = 
(P.4) {) if p <q. 
which is a (two-sided) multiplicative identity. An element f € /(P) has a (necessarily 


unique) left and right multiplicative inverse f—! if and only if f(p,p) 4 0 for all 
peEP. 


The zeta function and counting chains. An important element of /(P) is the zeta 
function 
C(p.q)=1 forall p<q. 
Notice that €*(p,q) is the number of multichains of length k from p to g. 
If P has a and 1, then 6 a (0,1) counts all multichains of length k — 2 in P, so the 
zeta polynomial and zeta function are related by 


Zp(k) =68(0,1) fork >2. (1.20) 


Similarly, (¢ —1)*(p,q) is the number of chains of length k from p to g. The sum 
1+(¢€—1)+(¢-1)?+--- has finitely many non-zero terms, and it equals (2— ¢)~!, 
where 2 = 2-1; so 


(2—¢)~'(p,q) = total number of chains in P from p to gq. 
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The Mobius function and Mo6bius inversion. The (equivalent) equations (1.18) 
and (1.19) defining the Mébius function can be rewritten as u¢ = 1 and Cu = 1, 
respectively. This explains why these two equations are equivalent: they say that 


i ae 


Proof of the Mobius inversion formula. _ Consider the left action of the incidence 
algebra I(P) on the vector space k” of functions f : P + k, given by 


a: f(p)= ¥ a(p.q)f (4) 


q2=P 


for a € I(P) and f : P > k. The Mobius inversion formula then states that g = ¢ - f 
if and only if 1. -g = f; this follows immediately from pp = €~!. a 


1.5.5.4 Computing Mobius functions 


In this section, we discuss some of the main tools for computing Mébius functions. 
Along the way, we prove the formulas for the Mobius functions of the posets of 
Theorem 1.5.15. 


1. (Chain n) We can check the formula for {yj manually. 


Mobius functions of products. A simple but important fact is that Mobius func- 
tions behave well under poset multiplication: 


Upxo((p,9),(p',4')) = Ue(p, Pp’) Ho(a, 7’): 


This is easily verified directly, and also follows from the fact that [(P x Q) = I(P) @x 
1(Q). 


2. (Boolean lattice 24) Since 24 Y 2 x --- x 2, the Mébius function of 2 tells us 
that the Mobius function of 24 is u(S,7) = (—1)!7~*. 


3. (Divisor lattice D,) Since Dy © (ty +1) x +++ x (tk +1) for n = p'} --- pi, we 
get the formula for Up, from the formula for the Mébius function of a chain. 


Mobius functions through Mobius inversion. So far we have thought of the 
Mobius function as a tool to apply Mobius inversion. Somewhat counterintuitively, 
it is possible to use Mobius inversion in the other direction, as a tool to compute 
Mobius functions. We carry out this approach to compute the M6bius function of the 
partition lattice I],,. 


4. (Partition lattice II,) Let II4 denote the lattice of set partitions of a set 
A ordered by refinement. Every interval [z,p] in II, is a product of par- 
tition lattices, as illustrated by the following example: In II9 we have 
[18|2|37|4|569, 13478|2569] = Ty1g.37,4, x Ty2,569; = Hs x Mo. Since the 
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Mébuus function is multiplicative, to compute the Mobius function of the par- 
tition lattices it suffices to show that uy, (0,1) = (—1)""!(n—1)}. 


Let W be the set of words w = w1 ...w, of length n in the alphabet {0,1,...,q— 
1}. Classify the words according to the equalities among their coordinates; 
namely, to each word w, associate the partition of [n] where i and j are in the 
same block when w; = wj. 


Let f(a) be the number of words whose partition is 2, and let g(z) be the 
number of words whose partition is a coarsening of 7. This is a situation where 
both f(z) and g(z) are easily computed: If z has b blocks then we have 


f(x) oe, : =q". 


Since g(x =. f(p) we get f(a = L(2,p)g(p). For x = 0 this says 
pen pen 
that m 
q(q—1)--(q-nt+1) = Y wO,p)q""l. 
pel 


Equating the coefficients of g! we get the desired result. 


Mobius functions through closures. A function ~ : P — P is a closure operator 
if 

e p<pPpforallp€P, 

e p<qimplies p <q for all p,q € P, and 

e p=Ppforall peEP. 


An element of p is closed if p = p; let Cl(P) be the subposet of closed elements of 
P. 


Proposition 1.5.18 [f= : P — P is a closure operator, then for any p <q in P, 


2 ee a .q) if pandqare closed 


otherwise. 


Proof. | We have an inclusion of incidence algebras /(C/(P)) + I(P) given by f> f 
where 
= f(p,q)_ if p and q are closed 
f(P:9) = 
0 otherwise. 


Let 1, ¢, and 7 be the image of the unit, zeta, and Mébius functions of Cl(P) in I(P). 
Also consider the “closure function” c € I(P) whose non-zero values are c(p,p) = 1 
for all p. Note that c€ = €1 because when q is closed, p < q if and only if p < q. 
Now we compute in J(P): 


ucl=uch=uClp=1p=p. 
This is equivalent to the desired equality. | 
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Mobius functions of lattices. When L is a lattice, there are two other methods 
for computing Mobius functions. The first one gives an alternative to the defining 
recursion for [1(p,q). This new recurrence is usually much shorter, at the expense of 
requiring some understanding of the join operation. 


Proposition 1.5.19 (Weisner’s Theorem) For any p <a <q ina lattice L we have 


Hu (p if ) =0. 
pSrsq:rVa=q 

Weisner’s theorem follows from Proposition 1.5.18 because r++ rV a is a closure 
operation, whose closed sets are the elements greater than or equal to a. There is also 
a dual version, obtained by applying the result to the reverse lattice L°P obtained from 
Lby reversing all order relations. 

Let us apply Weisner’s theorem to three additional examples of computing 
Mobius functions. 


5. (Subspace lattice L(F’)) Since an interval of height r in L(IF7) is isomorphic 
to L(G), it suffices to compute u(L(F7)) =: Mn. Let p = 0, q = 1, and let a 
be any line. The only subspaces r 4 1 with rVa=1 are the q" | hyperplanes 
not containing a. Each one of these hyperplanes satisfies [0,7] = L(t"), so 
Weisner’s theorem gives H,-+q"!U,—1 = 0, from which p, = (—1)"q""—)/2, 


6. (Partition lattice II,,, revisited) A similar (and easier) argument may be used 
to compute UW, = LU (II,), though it is now easier to use the dual to Weisner’s 
theorem. Let p = 0, q= ie and let a be the coatom 12...n—1|n. The only 
partitions r 0 with r \a = 0 are those with only one non-singleton block of 


the form {i,n} for i #n. Each such partition z has [7,1] = T,-1. The dual 
to Weisner’s theorem tells us that U, + (n—1)Un—1 = 0, from which U, = 


(-1)"(n—1)}. 


7. (Subgroup lattice L(G) of a finite p-group G) A similar argument may be used 
for the subgroup lattice L(G) of a p-group G. However, now one needs to 
invoke some facts about p-groups; see [210]. 


Theorem 1.5.20 (Crosscut Theorem) Let L be a lattice and let X be the set of atoms 


of L. Then Dit 
u(0, 1) = Y(-1)'M, 
k 


where Nx is the number of k-subsets of X whose join is 1, 


We will sketch a proof at the end of this section. Meanwhile, we point out a 
simple corollary of the Crosscut theorem: 


If the join of the atoms is not 1 then (0,1) =0. 
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8. (Distributive lattice L = J(P)) If J—J is an antichain in P, then [/,J] is a 
Boolean lattice in J(P) and uz (I,J) = (—1)/-7. Otherwise, the join of the 
atoms of [/,J] is 7 Umin(J —7) # J, and hence pi (7, J) = 0. 


9. (Young’s lattice Y) We obtain this Mébius function for free since Y is distribu- 
tive. 


Naturally, there are dual formulations to the previous two propositions, obtained 
by reversing the order of L. Also, there are many different versions of the crosscut 
theorem; see for example [169]. 


Mobius functions through multichains. If we know the zeta polynomial of a 
poset P with 0 and 1, we can obtain its Mébius number [1(P) immediately. 


Proposition 1.5.21 If P is a poset with 0 and 1, 
Zp(—1) = Ue (0,1). (1.21) 


Proof. We saw in (1.20) that Zp(k) = 6£(0, 1) for all integers k > 2. It would be 
irresponsible to just set k = —1, but it is very tempting, since €~! = py. 

In the spirit of combinatorial reciprocity, this is an instance where such irre- 
sponsible behavior pays off, with a bit of extra care. We know that Zp(k) is polyno- 
mial for k € Z, and we leave it as an exercise to show that ¢ «(0,1) is also polynomial 
for k € Z. Since these two polynomials agree on infinitely many values, they also 
agree for k = —1, and the result follows. | 


In light of (1.21), the zeta polynomial of P will give us the Mobius number pL (P) 
automatically. This is advantageous because sometimes the zeta polynomial is easier 
to compute than the M6bius function, as it is the answer to an explicit enumerative 
question. 


10. (Non-crossing partition lattice NC,,) From the zeta polynomial of NC, in Ex- 
ample 1.5.9 we immediately obtain that u(NC,) = (—1)"~'C,_1 where Cy 
is the (n — 1)th Catalan number. This gives a formula for the full Mébius func- 
tion, since every interval in NC, is a product of smaller non-crossing partition 
lattices. 


Mobius functions through topology. Equation (1.21) has a topological interpre- 
tation that is an extremely powerful method for computing Mobius functions. 


Proposition 1.5.22 (Phillip Hall’s Theorem) Let P be a finite poset with a0 and 1, 
and let c; be the number of chains 0= Po<pi<-<pi= 1 of length i from Otol 
in P. Then fade 

Lp(0,1) =co—cy +c2—-°:: 


This formula is equivalent to (1.21) in light of (1.17) and the relations bj = cj; + 
2cj—1 + cj—2; it may also be proved directly in the incidence algebra of P. 


90 Handbook of Enumerative Combinatorics 


Let us now interpret this result topologically. The order complex A(P) of a poset 
P is the simplicial complex whose vertices are the elements of P, and whose faces 
are the chains of P. 


Theorem 1.5.23 Let P be a finite poset with a0 and 1, and let P = P— {0, 1}. Then 


up(0,1) = x(A(P)) 


is the reduced Euler characteristic of A(P). 


Proof. In light of Proposition 1.5.22, this follows immediately from the combina- 
torial formula 


k=-1 


for the Euler characteristic of |A|, where f; is the number of k-dimensional faces 
of A. a 


Let us use this topological description to sketch proofs of the remaining Mobius 
functions of Theorem 1.5.15. 


11. (Face lattice of a polytope L(P)) The barycentric subdivision sd(P) is a sim- 
plicial complex with a vertex v(F’) at the barycenter of each proper face F. It 
has a simplex connecting vertices v(F),...,v(F,) whenever Fj C --- C Fy. As 
abstract simplicial complexes, sd(P) equals A(F (P)). Geometrically, sd(P) is 
a subdivision of the boundary of the polytope P, and hence is homeomorphic 


to the sphere S*™?-!, Therefore 
10,1) = Z(AP)) = ZS") = (-1) 


We will see in Section 1.6 that every interval of F(P) is itself the face poset of 
a polytope. It then follows that u(F,G) = (—1)mG-dimF 


12. (Face lattice of a subdivision 7 of a polytope P) A similar argument holds, 
though the details are slightly more subtle; see [194, Prop. 3.8.9]. 


13. (Bruhat order in permutations S,) There are several known proofs of the fact 
that (u,v) = (—1)!“)-!), none of which is easy. The first proof was an ad hoc 
combinatorial argument due to Verma [205]. Later Kazhdan and Lusztig [116] 
and Stembridge [195] gave algebraic proofs using Kazhdan-Lusztig polynomi- 
als and Hecke algebras, respectively. BjGrner and Wachs [40] gave a topolog- 
ical proof, based on Theorem 1.5.23 and further tools from topological com- 
binatorics. As we remarked earlier, there are similar formulas for the M6bius 
function of an arbitrary parabolic quotient of an arbitrary Coxeter group; see 
[38] or [195]. 


Theorem 1.5.23 tells us that in order to compute Mobius functions of posets of 
interest, it can be very useful to understand the topology of their underlying order 
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complexes. Conversely, combinatorial facts about Mobius functions often lead to the 
discovery of topological properties of these complexes. This is the motivation for the 
very rich study of poset topology. We refer the reader to the survey [206] for further 
information on this topic. 


1.5.6 Eulerian posets, flag f-vectors, and flag h-vectors 


Let P be a graded poset of height r. The flag f-vector (fs : S C [0,r]) and the flag 
h-vector (hs : SC [0,r]) are defined by 


fs = number of chains p; < po <--- < pg with {r(p1),...r(pe)} =S, and 


hs = ¥ (-1) 8" fy, fs= Yar 


TCS TCS 
for SC [0,7]. 


Example 1.5.24 Let H be a hexagonal prism of Figure 1.28, and let P = L(H) — 
{0,1} be its face lattice, with the top and bottom element (the empty face and the full 
face H) removed. It is not so enlightening to draw the poset, but we can still compute 
its flag f and h-vectors. For example, 19,2} = 36 counts the pairs (v, f) of a vertex v 
contained in a 2-face f. We get: 


6 O I 2 O1 02 12 O12 
fs|1 12 18 8&8 36 36 36 72 
hs |1 11 17 7 7 17 Il dt 


Figure 1.28 
A hexagonal prism. 


This example suggests that there may be additional structure in the flag h-vector; 
for instance in this example we have hs = hj q|_s for all S. This equation is not true 
in general, but it does hold for an important class of posets, which we now discuss. 
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Eulerian posets. Say a graded poset P is Eulerian if u(x,y) = (—1)"©)-’ for all 
x<y. 

In Theorem 1.5.15 we saw three important families of Eulerian posets: Boolean 
lattices, the Bruhat order, and face posets of polytopes. In particular, the poset P of 
Example 1.5.24 is Eulerian. 

For an Eulerian poset, there are many linear relations among the f5s, which are 
easier to describe in terms of the hss; for example, hs = hyo qs for all S. To describe 
them all, we further encode the flag f-vector in a polynomial in non-commuting 
variables a and b called the ab-index, which is defined to be 


p[a,b] = )° Asus 


SC [d] 


where us =u, ...ug and u; = aif i¢ S and u; = b if i € S. This is an element of the 
ring Z(a,b) of integer polynomials in the non-commutative variables a and b. 


Theorem 1.5.25 [28, 29, 191] The ab-index of an Eulerian poset P can be expressed 
uniquely as a polynomial in 


c=a-+b, d=ab+ba 


called the ed-index Yp(e,d) of P. Furthermore, if P is the face poset of a polytope, 
then the coefficients of the ed-index are non-negative. 


Example 1.5.26 The ab-index of the hexagonal prism in Example 1.5.24 is 


®(a,b) = aaa+1lbaa+17aba+ 7aab + 7bba+ 17bab + |labb + bbb 
= (a+b)? +6(a+b)(ab+ba) + 10(ab + ba) (a+b) 


so the cd-index is 


P(e,d) =e? + 6ed + 10de. 


In general, the eight entries of the flag f-vector of a 3-polytope (or any other Eule- 
rian poset of rank 3) are determined completely by only three numbers, namely, the 
coefficients of ced, and de in the ed-index. 


The cd-index encodes optimally the linear relations among the entries of flag f- 
vectors of polytopes. Since the number of monomials in c and d of ab-degree d is 
the Fibonacci number F,+1, this is the smallest number of entries in (fs) from which 
we can recover the whole flag f-vector—this is much smaller than 27. 


Corollary 1.5.27 [28, 29] The subspace of R2 spanned by the flag f-vectors of 
Eulerian posets of rank d (or by the flag f-vectors of d-polytopes) has dimension 
equal to the Fibonacci number Fy41. 
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PART 2. DISCRETE GEQMETRIC METHODS 


Part 2 is devoted to discrete geometry, which studies the connections between com- 
binatorics and the geometry of subspaces (points, lines, planes, ..., hyperplanes) in a 
Euclidean space. Configurations in the plane and 3-space have received great atten- 
tion, and feature many interesting results and open questions; however, very few of 
them involve exact enumeration in a meaningful way. Instead, we will focus on the 
discrete geometry of higher dimensions, where 


e studying “general” geometric configurations leads to interesting enumerative 
questions. 


e we have enough room to construct “special” geometric configurations that 
model various combinatorial structures of interest. 


This second part is divided into three sections on closely interrelated topics. In 
Section 1.6 we discuss polytopes, which are the higher dimensional generalization of 
polygons. Section 1.7 discusses arrangements of hyperplanes in a vector space. Fi- 
nally Section 1.8 is devoted to matroids, which are combinatorial objects that simul- 
taneously abstract arrangements of vectors, graphs, and matching problems, among 
others. 


1.6 Polytopes 


The theory of polytopes is a vast area of study, with deep connections to pure (alge- 
braic geometry, commutative algebra, representation theory) and applied mathemat- 
ics (optimization). Again, we focus on aspects related to enumeration. For a general 
introduction to polytopes, see [93, 225]. 

After discussing the basic definitions and facts in Section 1.6.1 and some im- 
portant examples in Section 1.6.2, we turn to enumerative questions. Section 1.6.3 
is devoted to the enumeration of faces of various dimensions. Section 1.6.4 is on 
Ehrhart theory, which measures polytopes by counting the lattice points that they 
(and their dilations) contain. 


1.6.1 Basic definitions and constructions 


Recall that a set S in Euclidean space R¢ is convex if for every pair of points u,v in 
S, the line segment uv is in S. The convex hull conv(S) of a set S C R¢ is the minimal 


convex set containing S. If S = {v1,...,v,} then 
conv{v,,...,Va} = {Avy t---+AnVn 2 A1,..-,An > 0,A, +++ +A, = 1} 
= intersection of all convex sets containing V],...,Vp- 


We will only be interested in convex polytopes, and when we talk about polytopes, 
it will be assumed that they are convex. 
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A hyperplane H in Euclidean space R¢ is an affine subspace of dimension d— 1; 
it is given by a linear equation 


H={xeR¢ :a-x=)} for some a € R4 — {0} and DER. 


It separates R@ into the two halfspaces given by the inequalities a-x < b anda-x > b, 
respectively. 

There are two equivalent ways of defining convex polytopes: The V-description 
gives a polytope in terms of its vertices, and the H-description gives it in terms of its 
defining inequalities. 


The V-description. A convex polytope P is the convex hull of finitely many points 
V1,.-.,V, in a Euclidean space R?: 


P =conv{vy,...,Vn} 


The H-description. A convex polytope P is a bounded intersection of finitely 
many halfspaces in a Euclidean space R?: 


—— {xeR’ : Ay X <1, XS by} 


= {xeR": Ax <b} 


where a1,..., Amn © IR¢ and by ,---;Dm € R. In the second expression we think of x as 
a column vector, A is the m x d matrix with rows a,,...a,, and b is the column vector 
with entries b1,...,Dm. 


Theorem 1.6.1 A subset P C R¢ is the convex hull of a finite set of points if and only 
if it is a bounded intersection of finitely many halfspaces. 


For theoretical and practical purposes, it is useful to have both the V-description 
and the H-description of a polytope. For example, it is clear from the H-description 
(but not at all from the V-description) that the intersection of two polytopes is a poly- 
tope, and it is clear from the V-description (but not at all from the H-description) that 
a projection of a polytope is a polytope. It is a nontrivial task to translate one descrip- 
tion into the other; see [225, Notes to Chapter 1] for a discussion and references on 
this problem. Here are some simple examples. 


Example 1.6.2 The standard simplex Ag_\, the cube Ug, and the crosspolytope 4 


are: 
1. Ag-1 = conv{ey,...,e7} 
= {xeR?:x,+---+xg=1andx;>O0fori=1.,...,d}, 
2. ad = conv{+e; +---+e, for any choice of signs} 
= {xeR?:-1<x;<1fori=1....,d}, 
3. Oa = conv{—ey,e),...,—e7,eq} 
= {xeR@: 4x, 4---+xy <1 for any choice of signs}, 
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respectively, where e;,...,@q are the standard basis in R¢. These polytopes are illus- 
trated in Figure 1.29 for d = 3. 


Figure 1.29 
The triangle Az, the cube U3, and the octahedron (3. 


The dimension of a polytope P is the dimension of the affine subspace spanned 
by P: 


aff(P) = {divi +o+ + ave DV1,---,Vk eR Aigo ERA + +&=0h. 


For example, even though Ag_ lives in R¢, its dimension is d — 1 because it lies on 
the hyperplane x; +---+2x%g = 1. 

The interior int(P) of a polytope P C R¢ is the topological interior of P. It is 
often more useful to consider the relative interior relint(P), which is its topological 
interior as a subset of the affine space aff(P). 


Polar polytopes. The similarity of our descriptions of Oy and $q is a manifes- 
tation of a general notion of duality between V-descriptions and H-descriptions of 
polytopes. 

Let P be a polytope in R? such that 0 € int(P). * The polar polytope P* of P is 


pA = {acR!:a-x<1 forallxe P}. 


If P has vertices vj,...v, and facets a; -x < 1,...,a,-x < 1 (and hence contains 0), 
then the polar P* has vertices a,,...@» and facets v;-x < 1,...,v,-x < 1. It follows 
that if 0 € P, then 

(PA)4 =P. 


Faces. For each vector a € R%, let 


Py={x€P:a-x>a-yforally <P} 


“We can apply an affine transformation to any full-dimensional polytope to make its interior contain the 
origin. 
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be the subset of P where the linear function a-x is maximized. Such a set is called 
a face of P. It is customary to allow the empty set to be a face of P as well. If 
dim(P) = d, then the faces of dimension 0,1,d —2,d — 1 are called vertices, edges, 
ridges, and facets, respectively. We let V(P) be the set of vertices of P. Faces other 
than P are called proper faces. We collect a few basic facts about faces. 


e A polytope is the convex hull of its vertices: P = conv(V (P)). 
e A polytope is the intersection of the halfspaces determined by its facets. 


e The vertices of a face F of P are the vertices of P contained in F: V(F) = 
V(P)OF. 


e A face F of P equals the intersection of the facets of P containing F’. 
e If F isa face of P, then any face of F is a face of P. 
e The intersection of two faces of P is a face of P. 


e A polytope is the disjoint union of the relative interiors of its faces: P = 


U relint(F’). 


F face 


The face lattice. The face lattice L(P) is the poset of faces of P, ordered by con- 
tainment. It is indeed a lattice with F \G = F NG and F VG =aff(FUG)NP. It is 
graded with rk(F’) = dimF + 1. We say that polytopes P and Q are combinatorially 
isomorphic if L(P) = L(Q). We collect some basic properties of face lattices: 


n~ 


e For each face F of P, the interval [0,F] of L(P) is isomorphic to the face 
lattice of F. 


~ 


e For each face F of P, the interval [F,1] of L(P) is isomorphic to the face 
lattice of a polytope, called the face figure P/F of F. 


e Every interval [F,G] of L(P) is isomorphic to the face lattice of some poly- 
tope. 


e The face lattice L(P4) of the polar polytope P is isomorphic to the opposite 
poset L(P)°?, obtained by reversing the order relations of L(P). 


There are different constructions of the face figure P/F, giving rise to combinato- 
rially isomorphic polytopes. We discuss one such construction. Let F° be the face of 
P* corresponding to the face F of P under the isomorphism L(P“) © L(P)°?. Then 
we can define the face figure to be the polar polytope of F°; that is, P/F = (F°)°. 
When F = vis a vertex, there is a more direct construction: let P/v = PMH, where 
H is a hyperplane separating v from all other vertices of P. 

We say a d-polytope P is simplicial if every face is a simplex. We say P is simple 
if every vertex is on exactly d facets (or, equivalently, on d edges). Note that the con- 
vex hull of generically chosen points is a simplicial polytope. Similarly, a bounded 
intersection of generically chosen half-spaces is a simple polytope. Also note that P 
is simplicial if and only if its polar P“ is simple. 
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Triangulations and subdivisions. In many contexts, it is useful to subdivide a 
polytope into simpler polytopes (most often simplices). It is most convenient to do it 
in such a way that the pieces of the subdivision meet face to face. 

A subdivision of a polytope P is a finite collection 7 of polytopes such that 


e P isthe union of the polytopes in 7, 
e ifP¢€ J thenevery face of P is in 7, and 
e ifP,QE FZ thnPNQE J. 


The elements of .7 are called the faces of .7; the full-dimensional ones are called 
facets. If all the faces are simplices, then .7 is called a triangulation of P. 

In some situations, it is useful to assume that a subdivision does not introduce 
new vertices; that is, that the only points in 7 are the vertices of P. We will assume 
that throughout the rest of this chapter. 


Theorem 1.6.3 Every convex polytope has a triangulation. 


Sketch of Proof. Let V be the set of vertices of our polytope P € R¢. For each 
height function / : V — R consider the set of lifted points P” = {(v,h(v)),v € V} 
in R¢*!, Let Q be the convex hull of P”, and consider the set Y of “lower facets” 
of Q that are visible from below, that is, the facets maximizing some linear function 
a-x with ag,, = —1. For each such facet F, let 2(F) be its projection back down to 
IR¢. One may check that {2(F) : F is a lower facet of Q} is a subdivision of P. If the 
height function is chosen generically, then this subdivision is actually a triangulation. 
See Figure 1.30 for an illustration. a 


Figure 1.30 
A regular subdivision of a hexagon. 


The subdivisions that may be obtained from the lifting construction above are 
called regular or coherent. In general, not every subdivision is regular, and the ques- 
tion of distinguishing regular and non-regular triangulations is subtle. In the simplest 
case, when P is a convex polygon, every subdivision is regular, and the number of 
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subdivisions is a Catalan number, as we saw in Section 1.3.2.2. Few other exact enu- 
merative results of this sort are known. 

In any case, regular subdivisions are of great importance for several reasons. 
One reason is that they are easy to define and construct; for example, choosing the 
heights h(v) = vj +---v? leads to the Delaunay triangulation, which is very easy 
to compute and has several desirable properties. Secondly, they have a very elegant 
structure. Every polytope P gives rise to a secondary polytope, whose faces are in 
bijection with the regular subdivisions of P; and faces F and G satisfy that F C Gif 
and only if the subdivision of F refines the subdivision of G. Thirdly, they are widely 
applicable, in particular, because they play a key role in the theory of Grébner bases 
in commutative algebra. For our enumerative purposes the regularity question will 
not be too important. For readers interested in this and other aspects of triangulations, 
we recommend [63, 87, 196]. 


1.6.2 Examples 


The following polytopes have particularly nice enumerative properties. We give ref- 
erences to some relevant results at the end of Section 1.6.4. 


1. (Product of two simplices) Ap_1 x Ag_1 = conv{(ej,fj) : 1<i<c,l1<j<d} 
in R° x R@ where {e,...,e-} and {f,,...,fz} are the standard bases for R° 
and R¢, respectively. 


2. (Hypersimplex) A(r,d) = conv{e;, +--- +e, : 1 <i) <---<i,<d}in R¢. 
3. (Permutahedron) 


TIg_; =conv{(a1,...,ag) : {a1,...,aqa} is a permutation of [d]}. 


4, (Zonotopes) The zonotope of a vector configuration A = {aj,...,a,} C R@ is 
the Minkowski sum Z(A) = [0,a;] +--- + [0,ay] := {Ajay +--+ +Agay 2 0 < 
Ay,---;Ax < 1}. The permutahedron ITj_; is the zonotope of the root system 
{ej—ej: 1<i<j<d}. 


5. (Cyclic polytope) Cy(m) = conv{(1,t;,t?,...,t7-!)|1 <i<m} fort) <--- < 


tm: 


6. (Order polytope of a poset P) O(P) = {x € R” : x; > 0 forall iand x; < 
x; ifi< jin P}. 


7. (Chain polytope of a poset P) 
€(P) ={x ER? : x; >0 for all i and 
Xj, +++++x;, < 1 for each chain i; <--- < ig in P}. 


8. (Type A root polytope) Ay_; =conv{e;—e; :1<iAj<d}inR4. 
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9. 


10. 


11. 


12. 


13. 


14. 


(Type A positive root polytope) Aa =conv{ej—ej;: 1<i<j<d}in R¢. 


(Flow polytope) Given a directed graph G = (V,E) and a vector b € RY, 
the flow polytope is Fg(b) = {f € R® : f. > 0 for alle € E and Ywwer fw — 
Yee fu = by for all v € V}. We think of f. as a flow on edge e, so that the 
excess flow or leak at each vertex v equals by. 


(CRY polytope) CRY, = Fx,,,(1,0,...,0,—1) where K,,1 is the complete 
graph on [n+ 1] with edges directed i + j fori < j. 


(Associahedron) Assocg_ is a polytope whose faces are in bijection with the 
ways of subdividing a convex (d +3)-gon into polygons without introducing 
new vertices. Faces F and G satisfy that F C G if and only if the subdivision 
of F refines the subdivision of G. There are several different polytopal realiza- 
tions of Assocg_1; see [50] for a survey. 


(Matroid polytope) If M = {v,,...,vg} is a set of vectors spanning a vec- 
tor space k’, the matroid (basis) polytope is Py = conv{ej, Pit eee: 
{vi,,.--;Vi,} is a basis of k’} in R¢. When M is generic we get Py = A(r,d). 
This construction is better understood in the context of matroids; see Section 
1.8. 


(Generalized permutahedra / polymatroids) Many interesting polytopes are de- 
formations of the permutahedron, obtained by moving the vertices of TIg_1 
while respecting all the edge directions. Examples include the polytopes Ag_1, 
Ae—1 X Ag—e-1, A(r,d), Tg-1, Ag_1, Pu, and Assocg_; above. Such poly- 
topes are called polymatroid base polytopes or generalized permutahedra; see 
[82, 158]. 


Many of these polytopes are related to the permutation group Sy and the corre- 
sponding type A root system conv{e;—e; : 1 <i j <d}. Many have generaliza- 
tions to the wider context of finite Coxeter groups and root systems; see for example 
[8, 42, 81, 83, 143] and the references therein. 

The faces of most of these polytopes can be described combinatorially. This 
should help us appreciate these polytopes, because there are not many interesting 
families of polytopes for which we can do this. We describe the most interesting 


ones. 


3. 


4. 


(Permutahedron) The H-description of ITg_; is 
Xp +++++x¢ =d(d+1)/2, 


Xi Ho Xi, >k(k+1)/2 for 0 © {i1,...,ix} C [d]. 


There is a bijection .Y «+ Fy between the ordered set partitions of [d] and the 
faces of the permutahedron Tg_}. 


(Zonotope) The face enumeration of Z(A) is the subject of the upcoming The- 
orem 1.7.15. 
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5. 


13. 


14. 
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(Cyclic polytope) Let t; = (1,#;,... ey aus The facets of the cyclic polytope 
Ca(m) are the simplices conv{t, : s € S} for the d-subsets S C [m] satisfying 
Gale’s evenness condition: Between any i < j not in S there is an even num- 
ber of elements of S. There is a similar description for all faces of the cyclic 
polytope. In particular, the combinatorics of Cy(m) is independent of t,...,tn- 
Two other remarkable facts are the following. 


e Every subset of at most d/2 vertices forms a face of Cq(m). 


e (McMullen’s Upper Bound Theorem [138]) Among all d-polytopes with 
m vertices, Cqg(m) maximizes the number of faces of dimension k for all 
2<k<d-l. 


The rich theory of positroids can be seen as a generalization of the study of 
cyclic polytopes; see Section 1.8 and [16, 157] for this connection. 


. (Order polytope) The vertices of G(P) are Lid, e: for the order ideals J C P. 


. (Chain polytope) The vertices of @(P) are Yj, e; for the antichains A C P. 


(Matroid polytope) The matroid polytope is cut out (non-minimally) by the 
inequalities 

ear |, yi xe <r(S) forE CS. 

ecE ecS 


The facets are characterized in [77]. 


(Generalized permutahedra) There are several interesting results on various 
classes of generalized permutahedra; see [156, 158]. 


1.6.3 Counting faces 


The f-vector of a d-polytope P is 


tp _ Gostiiseasieta) 


where f; is the number of (i — 1)-dimensional faces of P for 0 <i < d. Note that we 
include the empty face and omit the full-dimensional face P in this enumeration. The 
problem of characterizing the f-vectors of various kinds of polytopes (or more gen- 
eral polyhedral complexes) is a central one in combinatorics. We offer a very brief 
discussion; for more detailed accounts, see [33, 146, 225]. A one sentence summary 
is this: We completely understand the f-vectors of simplicial (or equivalently, of sim- 
ple) polytopes, but we are far from understanding the f-vectors of general polytopes. 
There are many interesting results and (mostly) open questions in between. 
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f-vectors. The most important result about f-vectors of arbitrary polytopes is the 
following. 


Theorem 1.6.4 (McMullen’s Upper Bound Theorem [138]) For any polytope P 
of dimension d with m vertices, we have 


fi(P) < fi(Ca(m)) fori=0,1,...,d—1, 


where Cq(m) is the cyclic polytope. 


h-vectors. If P is a simplicial d-polytope (or more generally, any simplicial com- 
plex of dimension d — 1), we define the h-vector hp = (hg, h1,...,ha) by the equiva- 
lent equations: 


hoxt + hyxt-) $+ + hgx® = fo(x—1)4 + fi(x—1)41 +--+ + fa(x—1)°, or 


i _jfd-j a ody 
m= Dew(4 Ni and one a for0 <i<d. 
, j=0 I—J 
The h-vector is a more economical way of storing the f-vector, due to the Dehn- 
Sommerville relations: hj = hg_; forO <i< d. 

In fact, the g-theorem characterizes completely the f-vectors of simplicial poly- 
topes [36, 188]! This spectacular result (which McMullen conjectured, and described 
as “even more intriguing, if rather less plausible” [138]) is one of the most important 
achievements of algebraic and geometric combinatorics to date. To state it, we need 
some definitions. 

We say a sequence of nonnegative integers (mo,771,...,1mq) is an M-sequence if 
there exists a set S of monomials in x),...,xX,, containing exactly m; monomials of 
degree i for i =0,1,...,d, such that m’ € S and m|m' implies m € S. Macaulay gave 
a numerical characterization, as follows. 

For any positive integers a and i, there is a unique representation a = (7) + 
(eh) fore @) for some aj > aj-| >--- > aj > j > 1. We then define a’) = 


(ae) + (41) +e + (ei. Then (7m, ,...,™mq) is an M-sequence if and only if 


mo = Land m1 < m\” fori =1,2,...,d—1. 


L 


Theorem 1.6.5 (Billera-Lee-Stanley’s g-theorem [36, 188]) A sequence (ho,...,ha) 
of positive integers is the h-vector of a simplicial d-polytope if and only if 


1. hj =hg_; for i=0,1,...,|d/2| (Dehn-Somerville equations), and 


2. (80,815+++,8|a/2|) is an M-sequence, where go = ho and g; = h; —hi-1 for 
1<i<|d/2|. 


For general polytopes, the situation is much less clear, even in dimension 4, a case 
that has been studied extensively. [226] Stanley [184] defined a more subtle toric h- 
vector. It coincides with the h-vector when P is simplicial, and it also satisfies the 
Dehn-Sommerville equations. His definition may be seen as a combinatorial formula 
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for the dimensions of the intersection cohomology groups of the corresponding pro- 
jective toric variety (if P is a rational polytope). In a different direction, for “cubical” 
polytopes P whose proper faces are all cubes, Adin [1] defined the cubical h-vector, 
which also satisfies the Dehn-Sommerville relations. Characterizing these toric and 
cubical h-vectors is an important open problem. 


Flag f-vectors and h-vectors. The flag f-vector (fp)pcjo,a—1| of a d-polytope 
enumerates the flags Fi C --- C F of given dimensions D = {di <--: < dg} for 
all D C [0,d — 1]. As we described in Section 1.5.6 (in the wider context of Eulerian 
posets), this information can be more economically stored in the cd-index of P. This 
encoding incorporates all linear relations among the flag f-vector. 

As we saw in Theorem 1.5.25, the cd-index of a polytope P is non-negative; this 
is not true for general Eulerian posets. Another very interesting question, which is 
wide open, is to classify the ed-indices of polytopes. 


1.6.4 Counting lattice points: Ehrhart theory 


In this section we are interested in “measuring” a polytope P by counting the lattice 
points in its integer dilations P,2P,3P,.... We limit our attention to lattice polytopes, 
whose vertices are lattice points, and to rational polytopes, whose vertices have 
rational coordinates; at the moment there is no good theory for general polytopes. 


Theorem 1.6.6 Let P C R¢ be a lattice polytope. There are polynomials Lp(x) and 
Lpo(x) of degree dimP, called the Ehrhart polynomial and interior Ehrhart poly- 
nomial of P, such that 


1. The number of lattice points in the nth dilation of P and its interior P° are 


Lp(n) = |nPNZ4| Lpo(n) = |nP° NZ4| for allneN. 


2. The Ehrhart reciprocity law holds: 


(—1)9™?Lp(—x) = Lpo(x). 


Note. If P is a rational polytope, then Lp(n) is instead given by a quasipolynomial; 
that is, there exist an integer m and polynomials L(x),...,Ln(x) such that we have 
|nP 1 Z¢| = L;(n) whenever n = k (mod m). 


Sketch of Proof of Theorem 1.6.6. By working in one dimension higher, we can 
consider the various dilations of P all at once. We embed R? into R¢+! by mapping 
v to (v, 1), and consider the cone 


cone(P) = {Ai (v1, 1) +---+An(Vn, 1): An,...,An > OF. 


Then for each n EN, the slice xg; =n of cone(P) is a copy of the dilation nP, 
as shown in Figure 1.31. Two key ingredients of the proof will be the lattice point 
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Figure 1.31 
The cone of a polytope and its slices at height 0, 1,2, 3. 


enumerator of cone(P) and the Ehrhart series of P: 


Ocone(P) (z) = } 2, Ehrp(z) = ) Lp(n)z" = Ocone(P) (1, ee 1,z), 
pécone(P) = co) 
4 (1.22) 
where zP = ag ae aa 


We first prove the theorem for simplices, and then use triangulations to prove the 
general case. 


Step 1. (Simplices) First we prove the result when P is a simplex, which we may 
assume is full-dimensional. In this case cone(P) is an orthant generated by d+ 1 
linearly independent vectors w),...,Wg41, where w; = (vj, 1). 


1. Define the lower and upper fundamental (half-open) parallelepipeds 
TE = {Aywit+-s++Ags wat 1 O<A1,...,Agzi <1} 
Wt = {Aiwy +--+ Ags was1 2 0<A1,...,Ag < I}. 


Then cone(P) is tiled by the various non-negative w-integer translates of IT, namely, 
the parallelepipeds II+ kiwi +--+ +ka41Wa+1 for k = (k1,...,ka41) € N@+!. Sim- 
ilarly, the interior int(cone(P)) is tiled by the various non-negative w-integer trans- 


lates of II+. Therefore 
1 1 
Toa) \ paren 


1 1 
— wi a ] — zWad+1 : 


Ont (1,..-,1,z) 
Ehrp (z) = ~ (d-get ? 


Ocone(P) (z) 


| | 
Ge g 

+ — 
——a N 

N ee 
wa ~~ 

i 
— 


Oint(cone(P)) (z) 


Using (1.22) we get 


on(1,...,1,z) 


Ehrp(z) = (l—z@! ; 
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which are rational functions. Then, by Theorem 1.3.6, Lp(n) and Lpo(n) are polyno- 
mial functions of n of degree d, as desired. 


2. Now observe that op+(z) = 2! + t4+1 og (1/z1,...,1/za41), because II* is 
the translation of —II by the vector w; + --- + Ww y+,. This gives 


Ehrpo (z) + (—1)4Ehrp(1/z) = 0. 


It remains to invoke the fact that if f is a polynomial, then F*(z) = L,sof(n)z” 
(which is a rational function of z) and F~(z) = Yyjeo f(n)z" (which is a rational 
function of 1/z, and hence of z) satisfy F*(z) + F(z) = 0 as rational functions. 
This implies that (—1)%"?Lp(—n) = Lpo(n) for every positive integer n, and hence 
these two polynomials are equal. 


Step 2. (The general case) Now let P be a general lattice polytope, and let 7 be a 
triangulation of P. Recall that 7° is the set of non-boundary faces of 7. 


1. We have 


Lp(n) = y Lro(n), Lpo(n) = y Lro(n) 


FeF FeZo 


which implies that Lp and Lpo are polynomials of degree dim P, by Step 1. 


2. Let TF be the face poset of .Y, with an additional maximum element 1. We 
have the Mobius dual relations 


Lr= y Lee, Lro = y L(G,F)L¢ forall FE FT 
G<F G<F 


omitting the arguments of the polynomials in question. This gives us 


Lp = SS Lro = y y L(G,F)Lg = y —u(G,1)L¢ = y Cpe any a, 


FeF FEZ G<F GET GeZ? 


where in the last step we are invoking Theorem 1.5.15.11. Now, using that the sim- 
plices G € 7° satisfy Ehrhart reciprocity, we obtain 


Lp(—n) = y (1 a lH) = (yee Lge (n) = (1) is (n), 
Ges? Ges? 


as desired. |_| 


Next we observe that the volume of a lattice polytope can be recovered from its 
Ehrhart polynomial. If dim P < d, there is a small subtlety: Let V be the affine span 
of P, and A = V UZ“. Then we need to normalize the volumes along V, so that any 
“unit” cube, generated by a Z-basis of A, has normalized volume 1. We let vol(P) 
denote the normalized volume or lattice volume of P. When P is full-dimensional, 
this is the usual volume. 

For example, the segment from (1,1) to (4,7) has normalized volume (lattice 
length) 3, because in this case the lattice A is generated by the primitive vector (1,2), 
and (4,7) — (1,1) =3(1,2). 
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Proposition 1.6.7 [f Lp(t) = cat? +---+ ct + co is the Ehrhart polynomial of a 
lattice polytope P, then cq = vol(P) and co = 1. 


Sketch of Proof. | We obtain better and better approximations of the volume of P by 


choosing finer and finer grids (42)4, and placing a cube of volume 5 centered at 


each lattice point in PM (42)4. Therefore 


d 
n 


Since the only lattice point in OP is the origin, co = Lp(0) = 1. a 


L 
nPniZ!| = lim 2a) =Cq. 


. 1 
vol(P) = tim a iim — 


= lim — 
n—oo nd 


A lattice simplex S in R@ with vertices v,,...,v, is unimodular if the vectors 
V2 —V],---,V¢g —V, are a lattice basis for the lattice aff SM Zt. A triangulation is 
unimodular if all its simplices are unimodular. The following result shows that uni- 
modular triangulations are particularly useful for enumerative purposes. 


Proposition 1.6.8 /. For every lattice polytope P C R4 the Ehrhart h*-polynomial 
h(x) = hp +hix+-+-+hix4, which is defined by 


hp(z) 
Ehrp(z) _ Ga 
has nonnegative coefficients: hi, > 0 for all k. 
2. If P has a unimodular triangulation 7, then 


hp(z) =hz(2), 


where the h-polynomial of 7 is the generating function hz(z) =ho+hyz+---+ 
haz for the h-vector (ho,...,hg) of 7, which is defined in terms of the f-vector 
(fo,---, fa) of F as in Section 1.6.3. 


Proof. 1. is due to Stanley [187]; for a short proof see [30, Theorem 3.12]. For 2., 


we have z= 
TIF (1,...,1,z 
Ehrp(z) = Yo Bhree(e)= Yo 
FeF FeZ (=z) ee 


where II, (F) is the upper fundamental parallelepiped of F. Since F is unimodular, 
the only lattice point in ig ks is the sum of its generators, which is at height dim F + 1, 


so 
dim F+1 d+l1 k d+1 k 
z z Lio hz 
pate Lime) = a5 
as desired. |_| 
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Examples. The following polytopes have particularly nice Ehrhart polynomials 
and Ehrhart series. 


1. (Polygon) P a lattice polygon in R?: 


B 508 I2+(I+B—3)z+1 
L =(I+=-1 ~n+1 Eh = 
p(n) ( 55 )n +5ntl, rp(z) (1-93 ; 
where P contains / lattice points in its interior and B lattice points on its bound- 
ary. This is equivalent to Pick’s formula for the area of a lattice polygon. 


2. (Simplex) Ag_1: 


La,_\(1) = ex a) Ehra,_,(z) = (2 


3. (Cube) Hy: 


_ LhoACd,k)z 
(1 — zat! i 


Lo, (n) = (n+1)%, Ehrp, (z) 


where the Eulerian number A(d,k) is the number of permutations 7 of [d] 
with k — 1 descents; that is, positions i with z(i) > z(i+1). 


4. (Crosspolytope) a: 
2 d\ (n (1+2z)4 
Zag(m) = 22 (;) @ Bho) = Cat 


5. (Product of two simplices) P = Ac_; x Ag_1: 


min(c—1,d—1) /c—1\ /d—1\ _k 
n+c—1\ /f(n+d-1 a z 
Lin) =( ie ( ae ), Ehrp(z) = 2&0 ee : 


Every triangulation of A-_; x Ag_; is unimodular, with h-vector given by the 
h*-vector above. These triangulations are very interesting combinatorially, and 
play an important role in tropical geometry and other contexts; see [10, 12, 66, 
175] and the references therein. 


6. (Hypersimplex) A(7,d): 


A(d—1,r) 


_ n+l 
(d—1)! 


Laiea (a) =" ( 


e vol(A;.¢) = 


where, again, A(d — 1,r) denotes the Eulerian numbers. No simple formula is 
known for the Ehrhart series of A(r,d). There is a nice formula for the “half- 
open hypersimplex’’; see [102]. An elegant triangulation of A(7,d) was given 
(using four different descriptions) by Lam-Postnikov, Stanley, Sturmfels, and 
Ziegler; see [127]. 
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7. 


10. 


11. 


(Permutahedron) P = ITq_): 
d-1 iS 
Ln,,(n)=¥ fin’, vol (Tg_1) = d*~ 
i=0 


where f; is the number of forests on [d] with i vertices. [190] 


. (Zonotope) P = Z(A): 


: 1 


where r is the rank of A and M(x,y) is the arithmetic Tutte polynomial of 
Section 1.8.9. [190] This polynomial is difficult to compute in general; when 
A is a root system, explicit formulas are given in [11]. 


. (Cyclic polytope) P = Cq(t),...,tm): 


d 
Lp(n) = y volC,(t1, bo stm nk. 
k=0 


See [130]. The triangulations of the cyclic polytope are unusually well be- 
haved; see [63, 163] and the references therein. In particular, when m = d +4, 
this is one of the few polytopes whose triangulations have been enumerated 
exactly (and non-trivially) [23]. 


(Order polytope and chain polytope) @(P),@(P): 
Lop) = Leip) =Qp(n+1), — vol(@(P)) = vol(@(P)) = e(P)/|PI!, 


where Qp is the order polynomial of P and e(P) is the number of linear exten- 
sions, as discussed in Section 1.5.3. [189] Remarkably, @(P) and @(P) have 
the same Ehrhart polynomial, even though they are not metrically, or even 
combinatorially, equivalent in general. There is a nice characterization of the 
posets P such that @(P) and @(P) may be obtained from one another by a 
unimodular change of basis [101]. 


(Root polytope) Ag_}: 


d  (d\2_% 
Ehra,(z) = ae vol(Aq) = 


d! 


There are similar formulas for the other classical root polytopes By,Cyg, Dag, as 
well as for the positive root polytopes. For example, vol(A7) = Cq/d! where 
Cq is the dth Catalan number. Explicit unimodular triangulations were con- 
structed in [8]. 
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12. (CRY polytope / Flow polytope) CRY; = Fx,,,, (1,0,...,0,—1): 
vol(CRY,,) = CoC C2-++Cn-2, 


a product of Catalan numbers. [223]. No combinatorial proof of this fact is 
known. Flow polytopes are of great importance due to their close connection 
with the Kostant partition function, which Gelfand described as “the transcen- 
dental element which accounts for many of the subtleties of the Cartan-Weyl 
theory” of representations of semisimple Lie algebras. [165]. There are many 
other interesting combinatorial results; see for example [25, 143]. 


13. (Matroid polytopes) A combinatorial formula for the volume of a matroid 
polytope is given in [9]. 


14. (Generalized permutahedra / polymatroids) There are many interesting results 
about the volumes and lattice points of various families of generalized permu- 
tahedra; see [158]. 


15. (Cayley polytope) C, = {x€ R" : 1 <x, <2 and 1 <x; <2x;_) for2 <i<n}: 
Cn+1 
vol(C,,) = a 

where c,, is the number of connected graphs on |n]. The related Tutte polytope 


has volume given by an evaluation of the Tutte polynomial (see Section 1.8.6) 
of the complete graph. [120] 


1.7. Hyperplane arrangements 


We now discuss arrangements of hyperplanes in a vector space. The questions that 
we ask depend on whether the underlying field is R,C, or a finite field F,; but in 
every case, the underlying combinatorics plays an important role. The presentation 
of this section is heavily influenced by [193]. See [153] for a great introduction to 
more algebraic and topological aspects of the theory of hyperplane arrangements. 

After developing the basics in Section 1.7.1, in Section 1.7.2 we introduce the 
characteristic polynomial, which plays a crucial role in this theory. In Section 1.7.3 
we discuss some of its important properties, and in Section 1.7.4 we develop several 
techniques for computing it. We illustrate these techniques by computing the charac- 
teristic polynomials of many arrangements of interest. Finally, in Section 1.7.5, we 
give a remarkable formula for the ed-index of an arrangement, which enumerates the 
flags of faces of given dimensions. 
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1.7.1 Basic definitions 


Let k be a field and V = k“. A hyperplane arrangement .°/ = {j,...,H,} is a 
collection of affine hyperplanes in V, say, 


H, ={x EV: v;-x=);} 


for nonzero normal vectors v1,...,V, € V and constants b1,...,b, € k. We say & is 
central if all hyperplanes have a common point; in the most natural examples, the 
origin is a common point. Figure 1.32 shows a central arrangement of four hyper- 
planes in R?. 


Figure 1.32 
A hyperplane arrangement. 


In some ways, central arrangements are slightly better behaved than affine ar- 
rangements. We can centralize an affine arrangement ./ in k” to obtain the cone of 
of, an arrangement c.o/ in k"*!, by turning the hyperplane a,x; +---+ay)X, =a in 
k” into the hyperplane a,x; +--+ +ayXp = aXy+1 in k"+! and adding the hyperplane 
Xnt1 = 0. 

Sometimes arrangements are “too central,” in the sense that their intersection is 
a subspace L of positive dimension. In that case, there is little harm in intersecting 
our arrangement with the orthogonal complement L+. We define the essentialization 
of & to be the arrangement ess(/) = {HNL+ : H € &#} in L+. The result is an 
essential arrangement, where the intersection of the hyperplanes is the origin. In 
most problems of interest, there is no important difference between & and ess(.2/). 

A key object is the complement 


V(a#) =V\ ( U n), 


Head 


and we now introduce a polynomial that is a fundamental tool in the study of V(.2). 
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1.7.2. The characteristic polynomial 


There is a combinatorial polynomial that knows a tremendous amount about the com- 
plement V (.%) of an arrangement /. The kinds of questions that we ask about V (.) 
depend on the underlying field. 


e If k =R then every hyperplane v;-x = b; divides V into two half-spaces, 
where v;-x < b; and v;-x > bj, respectively. Therefore an arrangement .@/ 
divides R¢ into a(.e/) regions, which are the connected components of 
the complement R@\./. Let b(.c/) be the number of those regions that are 
bounded. If is not essential, we let b(.) be the number of relatively 
bounded regions. These are the regions that become bounded in the essen- 
tialization ess(./). A central question about real hyperplane arrangements is 
to compute the numbers a(./) and b(.2/) of regions and bounded regions. 


e Ifk=C, then it is possible to walk around a hyperplane without crossing it; 
this produces a loop in the complement V(./). Therefore V(.) has nontriv- 
ial topology, and it is natural to ask for its Betti numbers. 


e Ifk =F, isa finite field, where q is a prime power, then V (./) is a finite set, 
and the simplest question we can ask is how many points it contains. 


Amazingly, the characteristic polynomial 7./(q) can answer these questions 
immediately. Let us define it. 

Define a flat of </ to be an affine subspace obtained as an intersection of hy- 
perplanes in </. The intersection poset L,, is the set of flats partially ordered by 
reverse inclusion. If ./ is central, then L,y is a geometric lattice, as discussed in 
Section 1.5.2. If & is not central, then Ly is only a geometric meet semilattice 
[207]. The rank r = r(./) of @ is the height of Ly. 


Figure 1.33 
The intersection poset of ./ and its Mobius function. 
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The characteristic polynomial of .& is 


ualg= ¥, uO,F)gt™’. 
Feely 


Figure 1.33 shows the intersection poset of the arrangement ./ in Figure 1.32; its 
characteristic polynomial y,y(q) = q° —4q* +5q —2 is easily computed by adding 
the Mobius numbers on each level of L,y. 

We will see some general techniques to compute characteristic polynomials in 
Section 1.7.4. 


Theorem 1.7.1 The characteristic polynomial 4 .7(x) contains the following infor- 
mation about the complement V (&/) of a hyperplane arrangement &/. 


1. (k=R, Zaslavsky’s Theorem) [220] Let &(A) be a real hyperplane arrange- 
ment in IR". The number of regions and relatively bounded regions of the com- 
plement V(&) are 


a(t) =(-1)"%e(-1), — () = (1) xar(1).. 


2. (k = C) [90, 152] Let e&/(A) be a complex hyperplane arrangement in C". 
The complement V(</) has Poincaré polynomial 


k he NN me. 
¥ rank H*(V(A),Z)qé = (4) xa ( ; ). 


3. (k = Fg) [20, 54] Let (A) be a hyperplane arrangement in Fi where F 4 is 
the finite field of q elements for a prime power q. The complement V(/) has 
size 


\V(A)| = Xx(Q). 


Proof. 3. This is a typical enumerative problem where our set of objects (the points 
of Fj) are stratified by a partial order, according to how special their position is with 
respect to <&. This is a natural setting to apply Mébius inversion. 

For each flat F, let f-(F) be the number of points of Fj which are on F, and on 
dim F 


no smaller flat. Since there are fs (F) =q points on F,, we have 


gut = > f_(G) 


G>F 


which inverts to 


Setting F = 0 gives the desired result. 

1. For each flat F let .& /F be the arrangement inside F obtained by intersecting 
the hyperplanes <7 — F with F. The arrangement subdivides each flat F into (rel- 
atively) open faces, namely, the (dimG)-dimensional regions of .//G for each flat 
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G > F. Since the Euler characteristic of F  R“™* is (—1)4'™", we get 
(-1yF = Y (-1)"™Fa(ar/G), 
G>F 


which inverts to 


(—1)°™*a(/F) = Y w(F,G)(-1)"°. 


Setting F = 0 gives the desired result. The same strategy works for b(.c/), using the 
result that the union of the bounded faces of .& is contractible, and hence has Euler 
characteristic equal to 1. [39, Theorem 4.5.7(b)] 

2. This proof is beyond the scope of this writeup; see [90, 153]. a 


1.7.3. Properties of the characteristic polynomial 


Whitney’s formula and the Tutte polynomial. Since the intersection poset . is 
constructed from its atoms (the hyperplanes of .2), it is natural to invoke the Crosscut 
Theorem 1.5.20 and obtain the following result. 


Theorem 1.7.2 (Whitney’s Theorem) The characteristic polynomial of an ar- 
rangement & is given by 


tag= Y (peg™M 


Bod 
B central 
where r(¥) =n—dimNye gl. 


The Tutte polynomial is another important polynomial associated to an arrange- 


ment: ae 
Ta(xyy= Yo @-1) 4) y—pi4r@, 
BA 
ZB central 


Whitney’s Theorem can then be rephrased as: 


X4a(q) = (-1)"q" "Taw — 4,0). 


1.7.3.1 Deletion and contraction 


A common technique for inductive arguments in hyperplane arrangement 7 is to 
choose a hyperplane H and study how . behaves without H (in the deletion ./\H) 
and how H interacts with the rest of . (in the contraction. # /H). 

For a hyperplane H of an arrangement . in V, the deletion 


@A\H ={AE af : AFH} 
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is the arrangement in V consisting of the hyperplanes other than H, and the contrac- 
tion 

A/H={ANH:AECA,AF#H} 
is the arrangement in H consisting of the intersections of the other hyperplanes with 


H. Figure 1.34 shows a hyperplane arrangement </ = {t,u,v,w} in R? and the dele- 
tion <\w and contraction & /w. 


ee 
ner 


Figure 1.34 
An arrangement and its deletion <\w and contraction & /w. 


Proposition 1.7.3 (Deletion-Contraction) [f </ is an arrangement and H is a hy- 
perplane in & then 


Xt (4) = Xan (9) — Lee /H(9)- 


Proof. Whitney’s formula gives 


swwg= Y (-VYiGers Yo Aye =yxnu(@) tan), 


H¢Bod HEB 
ZB central B central 
where we use the fact that if H € & then r(Y) =ryjy(A\H) +1. a 


Unfortunately hyperplane arrangements are not closed under contraction; in the 
example of Figure 1.34, the image of t in (.#/u)/v is not a hyperplane. Strictly 
speaking, this deletion-contraction formula applies only when there is only one copy 
of H in &. 

These small but annoying difficulties are elegantly solved by working in the wider 
class of matroids (for central arrangements; see Section 1.8) or semimatroids (for 
affine arrangements; see [7]). 
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The characteristic polynomial of the cone c. can be expressed as follows. 


Proposition 1.7.4 [fc is the cone of arrangement & then 


Neat (4) = (9-1) XQ). 


1.7.3.2 Sign alternation and unimodality 


More can be said about the individual coefficients of the characteristic polynomial. 
Theorem 1.7.5 The characteristic polynomial of an arrangement 
ter (9) =" — ang"! + ay—aq? = + (Nang? 


has coefficients alternating signs, so a; = 0. Furthermore, the coefficients are uni- 
modal and even log-concave, that is, 


a <a. <-++ SG <a> a4, >: > for some i, and 


aj-1aj+1 <a; for all j. 


Proof. The sign alternation is easily proved by induction using deletion- 
contraction. The second result is much deeper. It was conjectured by Rota in 1970 
[170], and recently proved by Huh [104] for fields of characteristic 0 and by Huh and 
Katz [105] for arbitrary fields, drawing from toric geometry, tropical geometry, and 
matroid theory. This result is conjectured to be true for any geometric lattice; this is 
still open. a 


1.7.3.3 Graphs and proper colorings 


There is a special case of interest, corresponding to arrangements coming from 
graphs. To each graph G on vertex set [n] we associate the graphical arrangement 
in k”, consisting of the hyperplanes x; = x; for all edges ij of G. 

Given gq colors, a proper g-coloring of G assigns a color to each vertex of G so 
that two vertices i and j that share an edge ij in G must have different colors. The 
number of proper g-colorings of G is given by the chromatic polynomial 7¢(q). 

Say an orientation of the edges of G is acyclic if it creates no directed cycles. 


Theorem 1.7.6 The chromatic polynomial of a graph G equals the characteristic 
polynomial of its graphical arrangement fg 


Ate(q) = X6(9); 


and the number of acyclic orientations of G equals the number of regions of & in 


R". 


Proof. A proper q-coloring of G is equivalent to a point a € F7 which is on none 
of the hyperplanes x; = x; for ij € G. Also, the arrangement .% in k” has the same 
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intersection lattice for any field k. (This is not true for every arrangement.) The first 
equality then follows from Theorem 1.7.1.3. 

Given a region R of & we give each edge ij the orientation i > j if x; > x; in 
R, and i + j if x; <x; in R. This is a bijection between the regions of % and the 
acyclic orientations of G. a 


In particular, this theorem proves that the chromatic polynomial 7%G(q) is indeed 
given by a polynomial in g when q is a positive integer. It also gives us a reciprocity 
theorem, telling us what happens when we irreverently substitute the negative integer 
q = —1 into this polynomial: 


Corollary 1.7.7 The graph G has |%g(—1)| acyclic orientations of G. 


1.7.3.4 Free arrangements 


Far more often than we might expect, characteristic polynomials of hyperplane ar- 
rangements factor as products of linear forms. [172] Such factorizations are often a 
manifestation of the underlying algebraic structure. The theory of free arrangements 
gives one possible explanation for this phenomenon. 

Let & be areal central arrangement and let R = R[x),...,x,] be the polynomial 
ring in n variables, graded by total degree. A derivation is a linear map D satisfying 
Leibniz’s law 

D(fg)=f(Dg)+(Df)g forall f,g ER. 


The set Der of derivations is an R-module; that is, if p € R and D € Der then 
pD € Der. It is a graded module, where D is homogeneous of degree d if it takes 
polynomials of degree k to polynomials of degree k+d. It is a free module with 
basis Fe Da? that is, 


Der = Bae + ee i; ER 
7 Pe a a a . 


Indeed, if D is a derivation with Dx; = p; € R for 1 <i<n, then D= p, a fener 
Pn a by linearity. 
Now consider the submodule of .-derivations: 


Der(/) = {D € Der : Gy divides D(a) for all H € A} 


where Qy is a linear form defining hyperplane H, so H = {v € k” : ay(v) =O}. Let 
Ow =Tlnew Ou be the defining polynomial of ./. 

For example, if # is the arrangement of coordinate hyperplanes with defining 
polynomial Ov = x1 --+X,, then E; = xige is an #-derivation for 1 <i<n since 
E;(x;) =X and Ej(x;) = 0 for J x i. 

We say the arrangement / is free if Der(.2) is a free R-module. This notion 
is not well understood at the moment; it is not even known if it is a combinatorial 
condition. For example, Ziegler gave an example of a free arrangement over F) and 
a non-free arrangement over F3 with isomorphic intersection posets. 
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Conjecture 1.7.8 [199] If two real arrangements &, and &, have isomorphic inter- 
section posets and &, is free, then & is free. 


Mysterious as it is, freeness is a very useful property. 


Theorem 1.7.9 (Terao’s Factorization Theorem [199]) If < is free then Der(&/) 
has a homogeneous basis D,...,D, whose degrees d,,...,d, only depend on &, 
and the characteristic polynomial of & is 


Xo (4) = (9-41 — 1) (Gdn - 1). 
Freeness is made more tractable thanks to the following two useful criteria. 


Theorem 1.7.10 (Saito Criterion [174]) Let D,,...,D, be &-derivations and let 
Qw be the defining polynomial of &. Then Der(@) is free with basis D,,...,Dpy if 
and only if 

det(Dj(xj))1<ij<n = ¢: Ou 


for some constant c. 


Theorem 1.7.11 [199] Let & be an arrangement and H be a hyperplane of &. Any 
two of the following statements imply the third: 


e &/H is free with exponents bj,...,by—1. 
e &\H is free with exponents bj,...,bn-1,bn— 1. 
e & is free with exponents b,,...,Dn—1,Dn 


Theorem 1.7.10 can be very easy to use if we have the right candidate for a basis. For 
example, we saw that E; = xige is an .%,-derivation for 1 <i<n. The matrix E;(x;) 
is diagonal with determinant x; ---x,, so this must in fact be a basis for Der(.#”) with 
exponents 0,...,0, and 

X6,(q) = (q—-1)". 

To use Theorem 1.7.11 to prove that an arrangement .& is free, we need & to 
belong to a larger family of free arrangements with predictable exponents, which 
behaves well under deletion and contraction. For example, to prove inductively that 
Hy, is free, we need the stronger statement that an arrangement of k coordinate hy- 
perplanes in k” (where k < n) has exponents 0 (k times) and —1 (n—k times). A 
more interesting example is given in Section 1.7.4. 


1.7.3.5 Supersolvability 


There is a combinatorial counterpart to the notion of freeness that produces similar 
results. Recall that a lattice L is supersolvable if it has an M-chain C such that the 
sublattice generated by C and any other chain is distributive. 

In this section we are interested in arrangements whose intersection poset is su- 
persolvable. Since this poset is a geometric lattice, it is semimodular, so the following 
theorem applies. 
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Theorem 1.7.12 [182] Let L be a finite supersolvable semimodular lattice, and sup- 
pose that0 =t90 <t) <+++ <t, =1 is an M-chain. Let a; be the number of atoms s 
such that s <t; but s mf t;_1. Then 


x14) = (G—41) +: (G—an). 


Graphical arrangements are an interesting special case. Say a graph G is chordal 
if there exists an ordering of the vertices vj,...,v, such that for each i, the vertices 
among {v1,...,¥j;-1} that are connected to v; form a complete subgraph. An equiva- 
lent characterization is that G has no induced cycles of length greater than 3. 

It is very easy to compute the chromatic polynomial of a chordal graph. Suppose 
we wish to assign colors from [g] to v1,...,V, in order, to get a proper q-coloring. 
If v; is connected to a; vertices among {v;,...,v;-1}, since these are all connected 
pairwise, they must have different colors, so there are exactly g— a; colors available 
for b;. It follows that yg(q) = (¢—a1)--:(q—a,). The similarity in these formulas 
is not a coincidence. 


Theorem 1.7.13 [182] The intersection lattice of the graphical arrangement &g is 
supersolvable if and only if the graph G is chordal. 


1.7.4 Computing the characteristic polynomial 


The results of the previous section, and Theorem 1.7.1 in particular, show the impor- 
tance of computing 7,,/(q) for arrangements of interest. In this section we discuss 
the most common techniques for doing this. 


Computing the Mobius function directly. In Section 1.5.5.4 we saw many tech- 
niques for computing Mébius functions, and we can use them to compute 7, (x). 
1. (Generic arrangement) %,,: _ generic hyperplanes in k’. 


Consider a generic arrangement of n hyperplanes in k”, where any k < r hy- 
perplanes have an intersection of codimension k. There are ( i flats of rank m, 


and for each flat F we have [0,F] & 2!"!, so u(0,F) = (—1)™. Therefore the 
characteristic polynomial is 


Koy (X) = y (-1yn( 2 


m=0 ve 


and the number of regions and bounded regions are 


atoms) = (")+(" )ane(), 
weed = ()-(h)e26)=(7) 


This method works in some examples, but when there is a nice formula for the 
characteristic polynomial, this is usually not the most efficient technique. 
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The finite field method. Theorem 1.7.1.3 is about arrangements over finite fields, 
but it may also be used as a powerful technique for computing 7.,(x) for real or 
complex arrangements. The idea is simple: Most arrangements .% we encounter “in 
nature” (that is, in mathematics) are given by equations with integer coefficients. 

We can use the equations of / to determine an arrangement .%, over a prime 
q. For large enough q, the arrangements «/ and .%, will have the same intersection 
poset, so 

to(9) =|F3\ U Al. 
HEA, 

We have thus reduced the computation of 7,v to an enumerative problem in a finite 
field. By now we are pretty good at counting, and we can solve these problems for 
many arrangements of interest. 


2. (Coordinate arrangement) 4,: x; =0 (1 <i<n) 
Here ¥.,(q) is the number of n-tuples (a1,...,@n) € Fj with a; £0 for all i, 
so 
X%,(9)=(4-1)", a Hn) = 2". 
There is an easy bijective proof for the number of regions: each region R of 
HA, is determined by whether x; < 0 or x; > Oin R. 


3. (Braid arrangement) %_—1: xj =x; (l<i<j<n). 
Here Y.v,_,(q) is the number of n-tuples (a1,...,4n) € FG such that a; 4 aj 


for i 4 j. Selecting them in order, a; can be any element of F, other than 
aj,...,Aj-|, SO 


Xet,.(9) =9(9—-1)(q—-2)---(q—n+1), — a(@H_-1) =n! 


A region is determined by whether x; > x; or x; < x; for all i ¥ j; that is, by 
the relative linear order of x),...,x,. This explains why the braid arrangement 
has n! regions. 


4. (Threshold arrangement) .%,: xj+x;=0, (l<i<j<n). 
Let q be an odd prime. We need to count the points a € F7 on none of the 
hyperplanes, that is, those satisfying a; +a; # 0 for all i ¢ j. To specify one 
such point, we may first choose the partition [n] = SoS) U---US, (q-1)/2 Where 
5; is the set of positions j such that a; € {i,—i}. Note that So can have at most 
one element. Then, for each non-empty block S$; with i > 0 we need to decide 
whether a; = i or aj = —i for all j € S;. The techniques of Section 1.3.3 then 


give 
n 


¥ ta 0)5 = (14221 Y?. 


The regions of %, are in bijection with the threshold graphs on [n]. These 
are the graphs for which there exist vertex weights w(i) for 1 <i <n and 
a “threshold” w such that edge ij is present in the graph if and only if w(i) + 
w(j) > w. Threshold graphs have many interesting properties and applications; 
see [135]. 


Algebraic and Geometric Methods in Enumerative Combinatorics 119 


There are many interesting deformations of the braid arrangement, obtained by 
considering hyperplanes of the form x; — x; =a for various constants a. The left 
panel of Figure 1.35 shows the braid arrangement .%. This is really an arrangement 
in R3, but since all hyperplanes contain the line x = y = z, we draw its essentialization 
by intersecting it with the plane x ++y-+z=0. Similarly, the other panels show the 
Catalan, Shi, Ish, and Linial arrangements, which we now discuss. 


OK SHE OK SIS 


Figure 1.35 
The arrangements .%, Catz, Shi, and Isho. 


5. (Catalan arrangement) Cat,_}: xj -—x; =—1,0,1 (l<i<j<n). 


To compute Yca,_,(q), we need to count the n-tuples a = (a1,...,dn) € i 
where a; and a; are never equal or adjacent modulo qg. There are g choices 
for a,, and once we have chosen a; we can “unwrap” F, — {a,} into a linear 
sequence of g—1 dots. The set A = {a2,...,a,} consists of n — 1 non-adjacent 
dots, and choosing them is equivalent to choosing a partition of g —n into n 
parts, corresponding to the gaps between the dots; there are Coa) choices. 
Finally there are (n — 1)! ways to place A in a linear order in a. Therefore 


Acat,_,(9) = 4(q—n—1)(q—n—2) ++ (q—2n+1) 
a(Caty_1) = n'Cn, b(Caty_1) — n'Cnh_-1, 


where C,, is the nth Catalan number. It is not too difficult to show bijectively 
that each region of the braid arrangement .%,_; contains C, regions of the 
Catalan arrangement, C,_; of which are bounded. [193, Section 5.4] 


6. (Shi arrangement) Shi,_;:  x;-—x; =0,1 (l<i<j<n). 

Consider a point a = (ay,...,dn) € Fy not on any of the Shi hyperplanes. Con- 
sider dots 0,1,...,g—1 around a circle, and mark dot a; with the number i 
for 1 <i<n. Mark the remaining dots e. Now let w be the word of length g 
obtained by reading the labels clockwise, starting at the label 1. Note that each 
block of consecutive numbers must be listed in increasing order. By record- 
ing the sets between adjacent es, and dropping the initial 1, we obtained an 
ordered partition IT of {2,3,...,n} into g—n parts. There are (q—n)"~! such 
partitions. Figure 1.36 shows an example of this construction. 
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_k 


2,1,3,0,8,4) —~ —— > 13600e25ee40e 


| 


2 6 (136,0,25,0,4) 


Figure 1.36 
a= (2,7,3,0,8,4) € F°,, w= 13600250040, Il = ({3,6},0, {2,5},0, {4}). 


To recover a from this partition, we only need to know where to put marking 
1 on the circle; that is, we need to know ay. It follows that 


Ashi, (9) =4(q—n)"" 
a(Shi,_;)=(n+1)""', —-b(Shi,_) = (n—-1)™"1. 


7. (Ish arrangement) Ishy_-1: xj=2j, x1 —xj=i (l<i<j<n). 


To choose a point a in the complement of the Ish arrangement over F,, choose 
a, € Fg, and then choose dy,@n—1,...,d2 subsequently. At each step we need 
to choose a; ¢ {a1,a; —1,...,4, —i +1,dn,an—1,..-,@i41}. These n forbidden 
values are distinct, so 


n—-1 


Xish,_1(4) = 4(q—n)", 
a(Ishp_1)=(n+1)""!, — b(Ishn_-1) = (n—1)""1. 


The Shi and Ish arrangements share several features, which are generally easier 
to verify for the Ish arrangement. Together, they give a nice description of the 
“q,t-Catalan numbers” [18]. 

8. (Linial arrangement) 1: xj—xj=1, (l<i<j<n). 
The Linial arrangement has characteristic polynomial 


14, =£¥ (Taw 


~9n 
2 k=0 


The number of regions of %,_1 equals the number of alternating trees with 
vertex set [n+ 1], where each vertex is either smaller than all its neighbors or 
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greater than all its neighbors. There are two different proofs, using Whitney’s 
Theorem [161] and the finite field method [22] respectively. Both are some- 
what indirect, combining combinatorial tricks and algebraic manipulations. To 
date, there is no known bijection between the regions of the Linial arrangement 
and alternating trees. 


Here are a few other nice examples. 


9. 


10. 


11. 


12. 


(Coxeter arrangement) B@,: xjtxj=0, x, =0 (l<i<j<n). 


To specify an n-tuples (a),...,dn) € Fi\Fen, we can choose a,...,dn SUC- 
cessively. We find there are g—2i+ 1 choices for a;, namely, any number other 
than 0,-+a,,...,-+a;_ (that are all distinct). Therefore 


XH6,(4) = (4-1)(q—3)---(q—2n+3)(q—2n+1), — r(BEn) = 2" -nl. 


(Coxeter arrangement) J,:  xjtxj =0 (l<i<j<n). 


Here there are (¢— 1)(¢—3)(q—5)---(1 — 2n +3)(q — 2n + 1) n-tuples in 
Fj\%n with no a; equal to 0, and n(q—1)(q—3)--+(q—2n+3)) n-tuples with 
one a; equal to 0. Therefore 


x9, (4) = (q—-1)(q—-3)-:+(q—2n+3)(q—nt1), — r(Jr) = 2"! -n! 


(Finite projective space) </(p,n): all linear hyperplanes in E 


The equations )';c;x; =0 (where c; € F,,) define an arrangement over F,, where 
q = p*, which has the same intersection poset. Recall that F, is a k-dimensional 


vector space over F’,,. Now let us count the points (a1,.--,@n) in F, that are not 
on any hyperplane. We choose aj,...,d, subsequently and at each step we 
need a; ¢ spany, (a1,...,@n—1). Therefore 


Aot(pn)(Q) = (@—1)(4— p)(Q—’)-+-(q—-p""'). 
See [27, 145]. 
(All-subset arrangement) All,: Yjcaxj =0 (A C |n], A #9, [n]). 


This arrangement appears naturally in combinatorics, representation theory, 
algebraic geometry, and physics, among many other contexts. To date, we do 
not know a simple formula for the characteristic polynomial, though we do 
have a nice bound. [32]. 


The reduction of All, modulo 2 is the arrangement ./(2,n) in F5 considered 
above. The map from All, to ./(2,n) changes the combinatorics; for example, 
the subset of hyperplanes x; + x2 = 0,x2 +.x3 =0,x3 +x, =0 decreases from 
rank 3 to rank 2. However, this map is a rank-preserving weak map, in the 
sense that the rank of a subset never increases, and the total rank stays the 


122 Handbook of Enumerative Combinatorics 


same. This implies [125, Cor. 9.3.7] that the coefficients of ¥au,, (q) are greater 
than the respective coefficients of ¥7(2,n) (q) in absolute value, so 


r(Alln) = [Xath(—1)] > Meran(—DI =F] @! +1) > 202) 


It is also known that r(All,) <2(-))”; see [32]. 


Whitney’s formula. We can sometimes identify combinatorially the terms in 
Whitney’s Theorem 1.7.2 to obtain a useful formula for the characteristic polyno- 
mial. 


13. (Coordinate arrangement) 4: x; =0 (1<i<n) 


Whitney’s formula gives 


talg= Y (lg 4 = (q-1)" 
AC[n] 


14. (Generic deformation of &%) Gi xj — xj = aij l<i<j< 
n, jj generic ) 
Let Hj; represent the hyperplane x; — x; = a;;. By the genericity of the a;;s, a 
subarrangement Hj, ;,,...,H;,;, 18 central if and only if the graph with edges 
ii j1,---5injn has no cycles; that is, it is a forest. Therefore 


x9,(q) = y (—1)FlgrlFI, r(G) = forests(n) 


F forest on [n] 


where forests() is the number of forests on vertex set [n]. There is no simple 
formula for this number, though we can use the techniques of Section 1.3.3 to 
compute the exponential generating function for 7y, (q). 


15. (Other deformations of .2%,) The same approach can work for any arrangement 
consisting of hyperplanes of the form x; — xj; = a;;, which correspond to edges 
marked a;;. If we have enough control over the a;;s to describe combinatorially 
which subarrangements are central, we will obtain a combinatorial formula for 
the characteristic polynomial. 


For simple arrangements like the Shi and Catalan arrangement, the finite field 
method gives slicker proofs than Whitney’s formula. However, this unified 
approach is also very powerful; for many interesting examples, see [161]. [6] 


Freeness. Terao’s Factorization Theorem 1.7.9 is a powerful algebraic technique 
for computing characteristic polynomials that factor completely into linear terms. 


16. (Braid arrangement, revisited) It is not difficult to describe the &%,_|- 


derivations for the braid arrangement .%,_,. Note that Fy = ae mn fee. +x! =o 
n 
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17. 


18. 


is an .%,_1-derivation for d =0,1,...,n—1 because Fy(x; —x;) ie = By 
Saito’s criterion, since 
det(Fi(x;))o<i,j<n—1 = det(x)i<ij<n= [] (i-*)) =e, 


1<i<j<n 
Der(.) is free with basis Fo,...,F,_1 of degrees —1,0,1,...,n—2. Therefore 
X0,(9) =9(q—-1)---(q-n+1). 


(Coxeter arrangements AG, and Y,, revisited) For Y,, the function Fy above 
satisfies Fa(xj-+xj) = xe +x4, so it is a Y,-derivation only for d odd. Fortu- 
nately, 
det(Fy:1(x,))i<ijen= [] Gi—x)Gitx)), 
l<i<j<n 
as can be seen by identifying linear factors as in Section 1.4.2. It follows that 
Der(Y,,) is free with basis F,, F3,Fs,...,Fo,—1 and degrees 0,2,4,...,2n —2. 


For BE, we need the additional derivation F = x, ---x%,¥; 1 2. This is in- 
L 1 


deed a BG ,-derivation because F (xj+x;) = (xj £xi)x1---Xn/xixj. Once again 
it is easy to check Saito’s criterion to see that Der(4@,) is free with basis 
F\,F3,...,Fon_3,F and degrees 0,2,4,--- ,2n —4,n—2. 


(Coxeter arrangements, in general) A finite reflection group is a finite group 
generated by reflections through a family of hyperplanes. For example the re- 
flections across the hyperplanes x; = x; of the braid arrangement .%,_1 corre- 
spond to the transpositions (ij), and generate the symmetric group S,. Every 
finite reflection group is a direct product of irreducible ones. Here we focus 
on the crystallographic ones, which can be written with integer coordinates. 
The irreducible crystallographic finite reflection groups ® come in three in- 
finite families A,,BC,,,D, and five exceptional groups Go, Fy, £6, £7, Es. The 
corresponding hyperplane arrangements .% are the following. 


k" | equations 
@ ” | equat 
By—\ | kk" | xp =x; 
BE |\k" | xi =txj, x, =0 
Dn kK" | xy = xy 
@% \1k3 |x = xj, 2xj =xXj+xq 
Fy |k* | x= £xj, x; =0, xy x. +x34x4 =0 
= 1l<a<3 
& ko | xp = Xj, 2x; = Xj (4<h<6) 
<c< 
ie {a,b,c} id{abc} Tses9 
18 AC{8 
& k° | xj = xj, be es (Ge) 
iGA i¢A 
. 1 
7.8 €=r 
& KY | Xj = £X;, y eixi =0 i ai) 
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19. 
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Note that arrangement ®, has rank r, but we have embedded some of them 
in k¢@ for higher d in order to obtain nicer equations. Here the indices range 
over the corresponding k”; for instance, % includes the hyperplanes 2x; = 
X2+x3, 2x2 =x, +x3, 2x3 =x] +xX2. We now list the characteristic polynomials 
and number of regions for these arrangements. 


@® | characteristic polynomial regions 
BE n | (a—1)(q—3)(q—5)-++ (q—2n+ 3) (q—2n+ 1) 2"n! 

D, = 1G=3) g=5)i(q—2at3)(g=n-1) 2 tn! 

% |(q-1W(q—5) 12 

Fa | (q—-1)(q—5)(q—7)(q— 11) 1,152 

& | (q—1)(q—4)(q—5)(q—7)(q—8)(q—11) 51,840 

& | (4—Wa—5)q—Tg—9)(q— 1) (q— 13)(q— 17) | 2,903,040 
6 | (q—1)(q—7)(¢q—11)(¢—13)(¢—17)(q—19) 696, 729, 600 

x(q— 23)(q— 29) 


It follows from the theory of reflection groups that the number of regions 
equals the order of the reflection group: 


r(@p) = |P|. 


We already computed in two ways the characteristic polynomials of the classi- 
cal Coxeter groups %,, 46n, and Y,. We first gave simple finite field proofs; 
it would be interesting to do this for the other (“exceptional”) groups. We 
also computed explicit bases for the modules of ./-derivations; doing this for 
the other Coxeter groups is not straightforward, but it can be done. [153, Ap- 
pendix B] 


However, these beautiful formulas clearly illustrate that there are deeper things 
at play, and they make it very desirable to have a unified explanation. Indeed, 
invariant theory produces an elegant case-free proof that all Coxeter arrange- 
ment are free; for details, see [153, Theorem 6.60]. 


(Shi arrangement) Given its characteristic polynomial, it is natural to guess that 
the Shi arrangement .%,_; is free. Since we only defined freeness for central 
arrangements, we consider the cone c.%,_; instead. Following [21], we may 
prove that this is a free arrangement inductively using Theorem 1.7.11. The 
strategy is to remove the hyperplanes of .%,_1 one at a time, by finding in 
each step a hyperplane H such that </\H and A/H are free with predictable 
exponents. This leads us to consider a more general family of arrangements. 


We claim that for any m > 0 and 2 <k <n-+1, the arrangement ve 


xj-xj = 0,1 for2<i<j<n 
xj—x; = O,1,...,m for2<j<k 
xj—-xj; = O,1,...,m,m+1 fork<j<n 
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is free with exponents n+m—1,....n+m—1(n—k-+1 times) ,n+m— 
2,...,n+m—2 (k—2 times), and —1. 


Obviously it takes some care to assemble this family of free arrangements 
and their exponents. However, once we have done this, it is straightforward 
to prove this more general statement by induction using Theorem 1.7.11, by 
removing one hyperplane at a time while staying within the family of arrange- 
ments gn. Setting k = 2 and m= 0 we get that the Shi arrangement is free. 


Incidentally, Athanasiadis also characterized all the arrangements < with 
By» CA C SF,_, such that ce is free [21]. These arrangements have Ish 
counterparts that have the same characteristic polynomial, although their cones 
are not free. [19, Corollary 3.3] 


1.7.5 The cd-index of an arrangement 


We now discuss a vast strengthening of Zaslavsky’s Theorem 1.7.1.1 due to Billera, 
Ehrenborg, and Readdy. [34] They showed that the enumeration of chains of faces 
of a real hyperplane arrangement .~ (or, equivalently, of a zonotope) depends only 
on the intersection poset of Ly. The equivalence with zonotopes is explained by the 
following result. 


Proposition 1.7.14 Let A be an arrangement of non-zero real vectors, and let & be 
the arrangement of hyperplanes perpendicular to the vectors of A. Then there is an 
order-reversing bijection between the faces of the zonotope Z(A) and the faces of the 
real arrangement &. 


Figure 1.37 
A vector arrangement, the normal hyperplane arrangement, and its dual zonotope. 


From this result, it follows easily that the ed-indices of Z(A) and F(./) are re- 
verses of each other; that is, they are related by the linear map * : Z(c,d) > Z(c,d) 
that reverses any word in c and d, so that (vj ...vg)* = vz...v1. To describe these ed- 
indices, we introduce the linear map @ : Z(a,b) — Z(c,d) obtained by first replacing 
each occurrence of ab with 2d, and then replacing every remaining letter with a c. 
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Theorem 1.7.15 [34] The flag f-vector of a real hyperplane arrangement (or equiv- 
alently, of the dual zonotope) depends only on its intersection poset. 

More explicitly, let o& be a real hyperplane arrangement. Let F(&) be its face 
poset and L,y be its intersection poset. The ed-indices of the face poset F(A) and 
the dual zonotope Z(@) are given in terms of the ab-index of the intersection lattice 
Ly by the formula: 


Prec) (€,d) = [@(a®,, (a,b))], Wr) (€,d) = o(a®y,, (a,b)). 


Billera, Ehrenborg, and Readdy [34] extended this result to orientable matroids, 
and asked for an interesting interpretation of @(a®,(a,b)) for an arbitrary geometric 
lattice L. 

To illustrate this result we revisit Example 1.5.24, where we computed the cd- 
index of a hexagonal prism. This is the dual zonotope to the arrangement of Figure 
1.37, whose intersection poset Ly is Figure 1.33. The flag f and h-vectors of Ly 
are (fo, fi}, fa} F123) = (1,4,4,9) and (ho, Ay ,442},4¢1,23) = (1,3,3,2), so its 
ab-index is ®; ,(a,b)) = aa+ 3ab + 3ba + 2bb. Therefore 


Pra) (¢,d) = @(aaa+3aab + 3aba + 2abb) = cee + 3e(2d) + 3(2d)c + 2(2d)e 
c? + 6ed + 10de 


and 
Vrs) (e,d) =e + 6de+ 10ed 


in agreement with the computation in Section 1.5.6. 


1.8 Matroids 


Matroid theory is a combinatorial theory of independence that has its roots in linear 
algebra and graph theory, but which turns out to have deep connections with many 
fields, and numerous applications in pure and applied mathematics. Rota was a par- 
ticularly enthusiastic ambassador: 


“Tt is as if one were to condense all trends of present day mathematics 
onto a single finite structure, a feat that anyone would a priori deem 
impossible, were it not for the fact that matroids do exist.” [171] 


There are natural notions of independence in linear algebra, graph theory, match- 
ing theory, the theory of field extensions, and the theory of routings, among others. 
Matroids capture the combinatorial essence that those notions share. 

In this section we will mostly focus on enumerative aspects of matroid theory. 
For a more complete introduction, see [154, 211] and the three volume series [214, 
215, 216]. 

In Section 1.8.1 we discuss some combinatorial aspects of vector configurations 
and linear independence, and introduce some terminology. This example and termi- 
nology strongly motivates the definition(s) of a matroid, which we give in Section 
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1.8.2. Section 1.8.3 discusses many important families of matroids arising in alge- 
bra, combinatorics, and geometry. Section 1.8.4 introduces some basic constructions, 
and Section 1.8.5 gives a very brief overview of structural matroid theory. The Tutte 
polynomial is our main enumerative tool; it is introduced in Section 1.8.6. In Sec- 
tion 1.8.7 we answer many enumeration problems in terms of Tutte polynomials, 
and in Section 1.8.8 we see how we can actually compute the polynomial in some 
examples of interest. Section 1.8.9 is devoted to two generalizations: the multivariate 
and the arithmetic Tutte polynomials. Finally, we give a brief overview of matroid 
subdivisions in Section 1.8.10, including a discussion on matroid valuations and the 
Derksen—Fink invariant. 


1.8.1 Main example: Vector configurations and linear matroids 


Let E C k¢ be a finite set of vectors. For simplicity, we assume that E does not 
contain 0, does not contain repeated vectors, and spans k“. The matroid of E is 
given by any of the following definitions: 


e The independent sets of E are the subsets of E that are linearly independent 
over k. 


e The bases of E are the subsets of E that are bases of k¢. 
e The circuits of E are the minimal linearly dependent subsets of E. 
e The rank function r : 2% — N is r(A) = dim(span A) for A C E. 


e The flats F of E are the subspaces of k@ spanned by subsets of E. We identify 
a flat with the set of vectors of E that it contains. 


e The lattice of flats Ly, is the poset of flats ordered by containment. 
e The matroid (basis) polytope is 
Py =conv{ep, +---+ep, : {bi,...,b;} isa basis} C R®, 


More concretely, if we know one of the following objects, we can determine them 
all: the list .% of independent sets, the list A of bases, the list @ of circuits, the rank 
function r, the list F of flats, the lattice Ly of flats, or the matroid polytope Py. 
For this reason, we define the matroid of E to be any one (and all) of these objects. 
A matroid that arises in this way from a set of vectors in a vector space (over k) is 
called a linear or representable matroid (over k). 


Example 1.8.1 Let t = (1,—1,0),u = (0,1,—1),v = (—1,0,1),w, = (1, 1,1) be the 
vector configuration shown in the left panel of Figure 1.37. Then 

= {0,t,u,v, w, tu, tv, tw, uv, uw, vw, tuw, tvw,uvw}, 

= {tuw,tvw,uvw}, 

{tuv} 


{0,t,u, v, w, tuv, tw, uw, vw, tuvw}. 


8 f B& 
| 
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(We are omitting brackets for clarity, so for example we write tuv for {t,u,v}.) The 
rank function is r(tuvw) = 3,r(tuv) = 2, and r(A) = |A| for all other sets A. The 
matroid polytope is the equilateral triangle with vertices (1,1,0,1), (1,0,1,1), and 
(0,1,1,1) in R*. 


A vector arrangement E C k@ determines a hyperplane arrangement / in k@ 
consisting of the normal hyperplanes {x € k? : a-x = 0} fora € E. The rank function 
and lattice of flats of E are the same as the rank function and intersection poset of ./, 
as defined in Section 1.7.2. For instance, the lattice of flats of the example above is the 
same as the intersection poset of the corresponding arrangement, which is illustrated 
in Figure 1.33. 


1.8.2 Basic definitions 


A simple but important insight of matroid theory is that many of the properties of 
linear independence (and many other notions of independence in mathematics) are 
inherently combinatorial. That combinatorial structure is unexpectedly rich, and ma- 
troids provide a general combinatorial framework that is ideally suited for exploring 
It. 

A matroid M = (E,-%) consists of a finite set E and a collection -¥ of subsets 
of E such that 

IbdeEc 4 

(12) If Je Y andI CJ then EZ. 

(13) If I,J € FY and |J| < |J| then there exists j ¢ J —J such that Uj € %. 
The sets in -¥ are called independent. 

The following result, which is an immediate consequence of (13), shows that this 
simple definition already captures the notion of dimension. 


Proposition 1.8.2 In a matroid M = (E,.#), all the bases (the maximal elements of 
JS ) have the same size, called the rank of the matroid. 


Now we develop the combinatorial versions of the definitions of the previous 
section. Let M = (E,-%) be a matroid. 


e An independent set is a set in 7. 
e A basis is a maximal independent set. 


e The rank function r : 2% > N is r(A) =(size of the largest independent sub- 
set of A). 


e A circuit is a minimal dependent set; i.e., a minimal set not in 7. 
e A flat is a set F such that r(F Ue) >r(F) for alle ¢ F. 
e The lattice of flats Ly, is the poset of flats ordered by containment. 


e The matroid (basis) polytope Py = conv{ep, +---+ep, : {b1,...,b;} isa 
basis}. 
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Again, we let Z, r, @ and ¥ denote the set of bases, the rank function, the set 
of circuits, and the set of flats of M. If we know the ground set E and any one of ¥, 
r, ©, F, Ly, or Py (assuming that we know the labels of the elements of Ly and 
the embedding of the polytope Py in R*), we know them all. Knowing E, we can 
recover the collection .% of independent sets from &, r, @, or ¥. For that reason, 
we call any one (and all) of (E,.%), (E,¥),(E,r), (E,@) and (E,.F) “the matroid 
M- Each one of these points of view has its own axiomatization. 


Proposition 1.8.3 We have the following characterizations of the possible bases Z, 
rank function r, circuits @, lattice of flats L, independent sets 4%, and matroid poly- 
topes Py of a matroid. 
1. The collection B © 2° is the set of bases of a matroid if and only if 
(BI) & is nonempty. 
(B2) If Bj,B2 © BZ and by © By — Bo, then there exists by © By — By such that 
(By —b,)Ub2 EB. 
2. The function r : 2 — N is the rank function of a matroid if and only if 
(RI)0<r(A) < |A| forallA CE. 
(R2) IfA CB CE, then r(A) < r(B). 
(R3) r(A)+r(B) > r(AUB)+r(ANB) for all A,B CE. 


3. The collection @ © 2° is the set of circuits of a matroid if and only if 
(C1) O€¢@. 
(C2) IfC €€ andC CD, thenD¢€ @. 
(C3) IfC,,Cy € © and e € Ci NC, then there exists C € © withC CC; UC) —e. 
4. The poset L is the lattice of flats of a matroid if and only if it is a geometric 
lattice. 
5. The collection % © 2¥ is the set of independent sets of a matroid if and only if 
(1)O0E F. 
(12) If] € % andI CJ thenleé F%. 
(13') For every weight function w: E — R, the following greedy algorithm 


produces a maximal element B of Y% of minimum weight w(B) = Ypcgw(b): 


Start with B = 0. Then, at each step, add to B an element e ¢ B of 
minimum weight w(e) such that BUe € -¥. Stop when B is maximal 


in F. 


6. [86] The collection BC 2® is the set of bases of a matroid if and only if 
every edge of the polytope conv{e,, +--+ +ep, : {b1,...,b,} € B} in R® isa 
translate of e; — e; for some i,j € E. 
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This proposition allows us to give six other definitions of matroids in terms of 
bases, ranks, circuits, flats, independent sets, and polytopes. The first three axiom 
systems are “not too far” from each other, but it is useful to have them all. For in- 
stance, to prove that linear matroids are indeed matroids, it is easier to check the cir- 
cuit axioms (C1)-(C3) than the independence axioms (I1)—-(13). The last three axiom 
systems suggest deeper, fruitful connections to posets, optimization, and polytopes. 

In fact, there are several other equivalent (and useful) definitions of matroids. It 
is no coincidence that matroid theory gave birth to the notion of a cryptomorphism, 
which refers to the equivalence of different axiomatizations of the same object. This 
feature of matroid theory can be frustrating at first; but as one goes deeper, it becomes 
indispensable and extremely powerful. A very valuable resource is [46], a multi- 
lingual cryptomorphism dictionary that translates between these different points of 
view. 

We say (E, @) and (E’, ¥’) are isomorphic if there is a bijection between E and 
E' that induces a bijection between @ and &’. A loop of M is an element e € E 
of rank 0, so {e} is dependent. A coloop is an element e that is independent of 
E —e, so r(E —e) =r—1. Non-loops e and f are parallel if r(ef) = 1; they are 
indistinguishable inside the matroid. A matroid is simple if it contains no loops or 
parallel elements. The map M ++ Ly is a bijection between simple matroids and 
geometric lattices. 


1.8.3. Examples 


We now describe a few contexts where matroids arise naturally. These statements are 
nice (and not always trivial) exercises in their respective fields. Most proofs may be 
found, for example, in [154]. We give references for the rest. 


0. (Uniform matroids) For n > k > 1, the uniform matroid U;,, is the matroid on 
[n] where every k-subset is a basis. It is the linear matroid of n generic vectors 
in a k-dimensional space. 


la. (Vector configurations) Let EF be a set of vectors in k? and let -F be the set of 
linearly independent subsets of E. Then (Z,.%) is a matroid. Such a matroid 
is called linear or representable (over k). [132, 217] There are at least three 
other ways of describing the same family of matroids, listed in 1b, Ic, and Id 
below. 


1b. (Point configurations) Let E be a set of points in k@ and let .¥ be the set of 
affinely independent subsets of E; that is, the subsets {a),...,a,} whose affine 
span is k-dimensional. Then (E, 7) is a matroid. 


lc. (Hyperplane arrangements) Let E be a hyperplane arrangement in k@ and let 
r({M,...,Ag}) =d—dim(A,N---OA;) for {M1,...,H,} C E. Then (E,r) is 
a matroid. 


1d. (Subspaces) Let V = k” be a vector space with a chosen basis, and let U C V 
be a subspace of codimension r. Consider the coordinate subspaces Vs = {v € 
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V : vs =0 for s € S}. Say an r-subset B C [d] is a basis if UNVg = {0}. If 
is the set of bases, (EZ, ¥) is a matroid. 


Given a matrix A over k, the matroid on the columns of A (in the sense of 1a) 
is the same as the matroid of the rowspace of A (in the sense of 1d). 


2. (Graphs) Let E be the set of edges of a connected graph G and let @ be the 
set of cycles of G (where each cycle is regarded as the list of its edges). Then 
(E,@) is a matroid. Such a matroid is called graphical. The bases are the 
spanning trees of G. 


3. (Field extensions) Let F C K be two fields, and let E be a finite set of ele- 
ments in the extension field K. Say J = {i1,..., ix} C E is independent if it is 
algebraically independent over F; that is, if there does not exist a non-trivial 
polynomial P(x;,...,x,) with coefficients in F such that P(i),...,i,) = 0. If 
SF is the collection of independent sets, (E,-%) is a matroid. Such a matroid is 
called algebraic. [204] 


4. (Matchings) Let G = (SUT,E) be a bipartite graph, so every edge in E joins 
a vertex of S and a vertex of T. Recall that a partial matching of G is a set 
of edges, no two of which have a common vertex. Let 7 be the collection of 
subsets J C T that can be matched to S; that is, those for which there exists 
a partial matching whose edges contain the vertices in J. Then (7,.%) is a 
matroid. Such a matroid is called transversal. [71] 


5. (Routings) Let G = (V,E) be a directed graph and Bo be an r-subset of the 
vertices. Let & be the collection of r-subsets of V for which there exists a 
routing (a collection of r vertex-disjoint paths) starting at B and ending at Bo. 
Then (V, ¥) is a matroid. Such a matroid is called cotransversal. [136] 


6. (Lattice paths) For each Dyck path P of length n, let B C [27] be the upstep set 
of P; that is, the set of integers i for which the ith step of P is an upstep. Let 
& be the collection of upstep sets of Dyck paths of length n. Then ([27],¥) 
is the set of basis of a matroid. This matroid is called a Catalan matroid 
C,,. More generally, we may consider the lattice paths of 2n steps (1,1) and 
(1,—1) which stay between an upper and a lower border; their upstep sets form 
the bases of a lattice path matroid. [4, 41] 


7. (Schubert matroids) Given n € N and a set of positive integers J = {i) <---< 
ix}, the sets {j) <--+ < jx} C [n] such that j; > i),..., 7, > ig are the bases of a 
matroid. These matroids are called Schubert matroids due to their connection 
to the Schubert cells in the Grassmannian Gr(k,n). This simple but important 
family of matroids has been rediscovered many times under names such as 
freedom matroids [52] and shifted matroids [4, 118]. 


8. (Positroids) A positroid is a matroid on {n] that can be represented by the 
columns of a full rank d x n matrix such that all its maximal minors are non- 
negative. [157] These matroids arise in the study of the totally nonnegative 
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part of the Grassmannian, and have recently found applications in the physics 
of scattering amplitudes. Remarkably, these matroids can be described combi- 
natorially, and they are in bijection with several interesting classes of objects 
with elegant enumerative properties. [17, 150, 157, 219] 


Je ID xipiTruti,. 445 


Figure 1.38 
Many manifestations of the same matroid. 


Example 1.8.4 The list above gives eight different manifestations of the matroid on 
{a,b,c,d,e, f} with bases B = {abc, abd ,abe,acd,ace}. Figure 1.38 shows a vector 
configuration in R> (with f =) that realizes M as a linear matroid, a graph that 
realizes it as a graphical matroid, a bipartite graph that realizes it as a transversal 
matroid, the five Dyck paths of length 3 that show that M is the Catalan matroid C3, 
and a directed graph with sinks By = {a,c,e} that realizes M as a cotransversal ma- 
troid. It can be realized as an algebraic matroid in the field extension R C R(x,y,z) 
witha =23,b=x+y,c =x—y,d =xy,e = xy’, f = 1. It is also isomorphic to 
the Schubert matroid on |6] for I = {2,4,6}. Finally, it is a positroid, since in the 
realization shown, all five bases are positive: They all satisfy the “right-hand rule” 
for vectors in R? when listed in alphabetical order. 


To give a small illustration of the power of matroids, let us revisit the almost 
trivial observation that all bases of a matroid have the same size. This result has 
many interesting consequences. 


e All bases of a vector space have the same size, called its “dimension.” 


e All spanning trees of a connected graph have the same number (v — 1) of 
edges, where v is the number of vertices of the graph. 


e Ina bipartite graph G = (SUT,E), all the maximal subsets of T that can be 
matched to S have the same size. 


e All transcendence bases of a field extension have the same size, called the 
“transcendence degree” of the extension. 
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Naturally, once we work harder to obtain more interesting results about matroids, we 
will obtain more impressive results in all of these different fields. 


1.8.4 Basic constructions 


We now discuss the notions of duality, minors, direct sums, and connected compo- 
nents in matroids; in Section 1.8.5 we explain how they generalize similar notions in 
linear algebra and graph theory. 


Proposition/Definition 1.8.5 (Duality) Let M = (E,&) be a matroid, and let B* = 
{E-—B: Be B}. Then M* = (E,&*) is a matroid, called the dual matroid of M. 


Proposition 1.8.6 For any matroid M we have (M*)* = M. 
Note that the loops of a matroid M are the coloops of M*. 


Proposition/Definition 1.8.7 (Deletion, contraction) Let M = (E,.%) be a matroid 
and s € E. Let 


I =e ss} gl! {I CE-s:IUsel}, if s is not a loop, 
= iS 5 = o . 
IF, if s is a loop. 


Then M\s = (E —s,.%') and M/s = (E —s,-%") are also matroids. They are called 
the deletion of s from M (or the restriction of M to E —s) and the contraction of s 
from M. 


Proposition/Definition 1.8.8 (Minors) Let M be a matroid on E. 


1. Deletion and contraction commute; that is, for all s #t in E, 
(M\s)\t =(M\t)\s, (M/s)/t=(M/t)/s, (M/s)\t = (M\t)/s. 


2. Let the deletion M\S (respectively, contraction M /S) of a subset S C E from M 
be the successive deletion (respectively, contraction) of the individual elements 
of S. Their rank functions are 


ry\s(A) =r(A), rm/s(A) =r(AUS) —r(S) (ACE-S). 
Let a minor of M be a matroid of the form M/S\T for disjoint S,T CE. 
3. Deletion and contraction are dual operations; that is, for all S C E, 
(M\S)* = M*/S. 
Proposition/Definition 1.8.9 (Direct sum, connected components) 


1. If M = (E,,-4%,) and Mg = (E>, -%2) are matroids on disjoint ground sets, then 
LF ={hUh: lh € 4,hb € 4} is the collection of independent sets of a 
matroid M, ® M2 = (FE; UE, ¥%), called the direct sum of M, and Mp. 
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2. Any matroid M decomposes uniquely as a direct sum M = M, © ---@M_. The 
ground sets of M,,...,M¢ are called the connected components of M. If c = 1, 
M is connected. 


The matroid polytope of M has dim(Py) = n—c where n is the number of ele- 
ments of M and c is the number of connected components. 


1.8.5 A few structural results 


Although they will not be strictly necessary in our discussion of enumerative aspects 
of matroids, it is worthwhile to mention a few important structural results. In particu- 
lar, in this section we explain the meaning of duality and minors for various families 
of matroids. 


Duality. 


e The dual of a linear matroid is linear. When M is the matroid of a subspace 
V of a vector space W, M* is the matroid of its orthogonal complement V+. 
For that reason, M* is sometimes called the orthogonal matroid of M. 


e The dual of a graphical matroid is not necessarily graphical; however, ma- 
troid duality is a generalization of graph duality. We say a graph G is planar 
if it can be drawn on the plane so that its edges intersect only at the endpoints. 
Such a drawing is called a plane graph. The dual graph G* is obtained by 
putting a vertex v* inside each region of G (including the “outside region’), 
and joining v* and w* by an edge labeled e* if the corresponding regions of 
G are separated by an edge e in G. This construction is exemplified in Figure 
1.39. The graph G* depends on the drawing of G, but its matroid does not: if 
Gis planar, we have M(G*) = M(G)*. 


Figure 1.39 
A plane graph G and its dual G*. 


e Embarrassingly, no one knows whether the dual of an algebraic matroid is 
always algebraic. This illustrates how poorly we currently understand this 
family of matroids. 
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Cotransversal matroids are precisely the duals of transversal matroids. 


The Catalan matroid is self-dual. The dual of a lattice path matroid is a lattice 
path matroid. 


Linear programming duality can be framed and generalized in the context of 
oriented matroids; see [39] for details. 


Corollary 1.8.10 Jf a connected planar graph has v vertices, e edges, and f faces, 
thenv—e+f =2. 


Proof. _ A basis of our graph G has v— 1 elements and a basis of its dual graph 
G* has f —1 elements; so if M is the matroid of G, we have (v—1)+(f—1) = 
r(M)+r(M*) =e. a 


Minors. 


The minors of a linear matroid are linear. If M is the matroid of a vector 
configuration E C V and ve E, then M\v is the matroid of E\v, and M/Vv is 
the matroid of E\v modulo y; that is, the matroid of 7(E\v), where 7: V > 
V/(spanv) is the canonical projection map. 


The minors of a graphical matroid are graphical. If M is the matroid of a 
graph G and e = wv is an edge, then M\e is the matroid of the graph G\e 
obtained by removing the edge e, and M/e is the matroid of the graph G/e 
obtained by removing the edge e and identifying vertices u and v. 


[154, Cor. 6.7.14] The minors of an algebraic matroid are algebraic. 


Transversal matroids are closed under contraction, but not deletion. Co- 
transversal matroids are closed under deletion, but not contraction. A gam- 
moid is a deletion of a transversal matroid or, equivalently, a contraction of 
a cotransversal matroid. Gammoids are the smallest minor-closed class of 
matroids containing transversal (or cotransversal) matroids. 


[41] Catalan matroids are not closed under deletion or contraction. Lattice 
path matroids are closed under deletion and contraction. 


Direct sums and connectivity. 


If M, and M) are the linear matroids of configurations E; C V; and E2 C V2 
over the same field k, then M; ® Mp is the matroid of {(v1,0) : v;) € Ei: }U 
{(0,v2) : Vo € Eo} inV, @Vd. 


If M,; and Mp) are the matroids of graphs G; and G2, then M; & Mp is the 
matroid of the disjoint union of G; and G2. The matroid of a loopless graph 
G is connected if and only if G is 2-connected; that is, G is connected, and 
there does not exist a vertex of G whose removal disconnects it. 
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Non-representability. Although it is conjectured that almost all matroids are not 
linear and not algebraic, it takes some effort to construct examples of non-linear and 
non-algebraic matroids. We describe the simplest examples. 


Figure 1.40 
The Fano matroid, the non-Fano matroid F; , and the Vamos matroid Vs. 


e The Fano matroid F7 is the linear matroid determined by the seven non- 
zero vectors in F3. This matroid is illustrated in the left panel of Figure 1.40, 
where an element is denoted by a vertex, and a circuit is denoted by a line or 
circle joining the points in question. The Fano matroid is representable over 
k if and only if char(k) = 2. 


e The non-Fano matroid F, is the linear matroid determined by the seven 
non-zero 0-1 vectors (0,0, 1),(0,1,0),(1,0,0), (0,1, 1),(1,0,1),(1,1,0), and 
(1,1,1) in R?. It is representable over k if and only if char(k) 4 2. 


e The direct sum F7 @ F; is not representable over any field. 


e The Vamos matroid Vs is a matroid of rank 4 whose only non-trivial circuits 
are 1234, 1456, 1478, 2356, 2378, as illustrated in the right panel of Figure 
1.40. This matroid is not representable over any field, because in any affine 
configuration of points with these coplanarities, 5678 would also be coplanar. 
The Vamos matroid is also not algebraic over any field. 


Relating various classes. Most matroids that we encounter naturally in combina- 
torics are linear, although this is usually not clear from their definition. 


e All the examples of Section 1.8.3 are linear, with the possible exception of 
algebraic matroids. 


e Graphical, transversal, cotransversal matroids, and positroids are linear. 


e Linear matroids over k are algebraic over k. 
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e Algebraic matroids over k of characteristic 0 are linear over a field extension 
of k. 


e Catalan matroids are Schubert matroids. 
e Schubert matroids are lattice path matroids. 
e Lattice path matroids are transversal. 


e Positroids are gammoids. 


Obstructions to representability. A general question in structural matroid theory 
is to try to describe a family .¥ of matroids by understanding the “obstructions” 
belonging to ¥. These results are inspired by Kuratowski’s Theorem in graph theory, 
which asserts that a graph is planar if and only if it does not contain the complete 
graph Ks or the complete bipartite graph K33 as a minor. Similarly, one seeks a 
(hopefully finite) list of minors that a matroid must avoid to be in ¥. 

The question that has received the most attention has been that of characterizing 
the matroids that are representable over a particular field. Here are some results: 


e Mis representable over F if and only if it has no minor isomorphic to U2 4. 


e M is representable over F3 if and only if it has no minor isomorphic to 
U2,5,U3,5,F7, or F;. 


e M is representable over every field if and only if it has no minor isomorphic 
to U24,F), or Fe. 


e Mis graphical if and only if it has no minor isomorphic to U2,4,F7,F,M(Ks)*, 
or M(K33 ) es é 


e Forany field F of characteristic zero, there are infinitely many minor-minimal 
matroids that are not representable over F. 


e (Rota’s Conjecture) For any finite field F,, there is a finite set .%, of ma- 
troids such that M is representable over F, if and only if it contains no minor 
isomorphic to a matroid in 7. 


In 2013, Geelen, Gerards, and Whittle announced a proof of Rota’s conjecture. In 
[85] they write: “We are now immersed in the lengthy task of writing up our results. 
Since that process will take a few years, we have written this article offering a high- 
level preview of the proof.” 


Asymptotic enumeration. The asymptotic growth of various classes of matroids 
is not well understood. Let P be a matroid property, and let p(n) be the number 
of matroids on [n] having property P. Let m(n) be the number of matroids on [n]. If 
lim, 00 p(n) /m(n) = 1, we say that asymptotically almost every matroid has property 
P. We list some conjectures. [137] 
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e Asymptotically almost every matroid is connected. 


e Asymptotically almost every matroid M is paving; that is, all its circuits have 
sizerorr+1. 


e Asymptotically almost every matroid is asymmetric, that is, it has no non- 
trivial automorphism. 


e Asymptotically almost every matroid is not linear. 


e Asymptotically almost every matroid of n elements on rank r satisfies the 
inequality (n—1)/2<r<(n+1)/2. 


We also list some results. 


e [26, 119] The number m(n) of matroids on [n] satisfies 


3 1 2 3 1 2 
n 5 logn+ 5 log = o(1) <loglogm(n) <n 5 logn4 5 log - +1+o(1). 


e [137] Asymptotically almost every matroid is loopless and coloopless. 


e [137] Asymptotically, the proportion of matroids that are connected is at least 
1/2. (It is conjectured to equal 1.) 


e [17] Asymptotically, the proportion of positroids that are connected is 1 /e?. 


1.8.6 The Tutte polynomial 


Throughout this section, let R be an arbitrary commutative ring. A function f : 
Matroids — R is a matroid invariant if f(M) = f(N) whenever M = N. Let L and 
C be the matroids consisting of a single loop and a single coloop, respectively. 


1.8.6.1 Explicit definition 
The invariant that appears most often in enumerative questions is the Tutte polyno- 


mial 


Tu(x,y) = Y (@e-1yr "4 (yy) 4Alrr@), (1.23) 
ACM 


When we meet a new matroid invariant, the first question we usually ask is whether 
it is an evaluation of Thy (x,y). 
1.8.6.2 Recursive definition and universality property 


The ubiquity of the Tutte polynomial is not an accident: it is universal in a large, 
important family of matroid invariants. Let us make this precise. Let R be a ring. 
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Say f : Matroids > R is a generalized Tutte-Grothendieck invariant if for every 
matroid M and every element e € M, 


af(M\e)+bf(M/e) if e is neither a loop nor a coloop, 
f(M) = ¢ f(M\e)f(L) if e is a loop, (1.24) 
f(M/e)f(C) if e is a coloop, 


for some non-zero constants a,b € R. It is a Tutte-Grothendieck invariant if a = 
b=1. 


Theorem 1.8.11 The Tutte polynomial is a universal Tutte-Grothendieck invariant; 
namely: 


1. For every matroid M and every element e € M 


Tu\e(*,¥) + Tuje(x,y) if e is neither a loop nor a coloop, 
Tu (x,y) = yTy\e(%,y) if e is a loop, 
xTy je(x,y) if e is a coloop. 


2. Any generalized Tutte-Grothendieck invariant is a function of the Tutte poly- 
nomial. Explicitly, if f satisfies (1.24), then 


f(M) =a" "b'Ty (40,0) 


where n is the number of elements and r is the rank of M. We do not need 
to assume a and b are invertible; when we multiply by a"~" b’, we cancel all 
denominators. 


Sketch of Proof: This is a very powerful result with a very simple proof. The first 
part is a straightforward computation from the definitions. The second statement then 
follows easily by induction on the number of elements of M. | 


More generally, write T(x,y) = Yj ;tijx'y/. If f satisfies f(M) = f(M\e) + 
f(M/e) when e is neither a loop nor a coloop, then f(M) = Y; jtijf(C' @L’), where 
C' @L/ denotes the matroid consisting of i coloops and j loops. 


Proposition 1.8.12 The Tutte polynomial behaves well with respect to duality and 
direct sums: 


1. For any matroid M, Ty=(x,y) = Tu(y,x). 


2. For any matroids M and N on disjoint ground sets, Tyen(x,y) = 
Tu (x,y) Tw (x,y). 
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1.8.6.3 Activity interpretation 


An unexpected consequence of Theorem 1.8.11 is that the Tutte polynomial has non- 
negative coefficients; this is not at all apparent from the explicit formula (1.23). As 
usual, the natural question for a combinatorialist is: What do these coefficients count? 
The natural question for an algebraist is: What vector spaces have these coefficients 
as their dimensions? At the moment, the first question has a nice answer, while the 
second one does not. 

Fix a linear order < on the elements of E. Say an element i € B is internally 
active if there is no basis B—iUj with j < i. Say an element j ¢ B is externally 
active if there is no basis B —iU j with i < j. These are dual notions: i is internally 
active with respect to basis B in M if and only if it is externally active with respect to 
basis E — B in M*. 


Theorem 1.8.13 [53, 201] For any linear order < on the ground set of a matroid, 
let I(B) and E(B) be, respectively, the set of internally active and externally active 
elements with respect to B. Then 


Tulx.y) = YL V@ylEeI, 


B basis 


Sketch of Proof. It is possible to give a standard deletion-contraction proof, by 
showing that the right-hand side is a Tutte-Grothendieck invariant, and applying 
Theorem 1.8.11. However, there is a much more enlightening explanation. One may 
prove [53] that the intervals [B\J(B),BUE(B)] partition the Boolean lattice 2%. In 
other words, every subset of the ground set can be written uniquely in the form 
(B\J) UF where B is a basis, J C I(B), and F C E(B). Furthermore, under these 
assumptions r((B\J) UF) = r—|J|. Then (1.23) becomes 


= yy d IMliy —1)Fl= y WB) yIE@)| 


B basis JCI(B) FCE(B B basis 


as desired. |_| 


This combinatorial interpretation of the non-negative coefficients of the Tutte 
polynomial is very interesting, but it would be even nicer to find an interpretation 
that does not depend on choosing a linear order on the ground set. 

In a different direction, Procesi [60] asked the following question: Given a ma- 
troid M, is there a natural bigraded algebra whose bigraded Hilbert polynomial is 
Tu (x,y)? This question is still open. It may be more tractable (and still very interest- 
ing) when M is representable; we will see some approximations in Section 1.8.7. 


1.8.6.4 Finite field interpretation 


The coboundary polynomial ¥,,(X,Y) is the following simple transformation of 
the Tutte polynomial: 
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It is clear how to recover T (x,y) from 7(X,Y). We have the following interpretation 
of the coboundary (and hence the Tutte) polynomial. 


Theorem 1.8.14 (Finite Field Method [6, 54, 213]) Let < be a hyperplane ar- 
rangement of rank r in Fé. For each point p € Fa let h(p) be the number of hy- 
perplanes of & containing p. Then 


yr rh) X(qst)- 


peFd 


Proof. For each p € F@, let H (p) be the set of hyperplanes of ./ that p lies on. 
Then 


¥ (q,t) =: y Yale 14 = y qe aa 
‘ BOA BA 
= YY |nar-yi4t=Y YY ¢-1)4 
Bod BCA pEnB 
= Yen = F 146-1), 
pekj @CH(p) perky 
as desired. |_| 


Computing Tutte polynomials is extremely difficult (#P-complete [212]) for gen- 
eral matroids, but it is still possible in some cases of interest. Theorem 1.8.14 is one 
of the most effective methods for computing Tutte polynomials of (a few) particular 
arrangements ./ in k“, as follows. 

If the hyperplanes of .&/ have integer coefficients (as most arrangements of inter- 
est do), we may use the same equations to define an arrangement .%, over Fa. Ifgisa 
power of a large enough prime, then ./ and .«, have isomorphic matroids, and hence 
have the same Tutte polynomial. Then Theorem 1.8.14 reduces the computation of 
T.z(x,y) to an enumerative problem over F4 , which can sometimes be solved. [6] 


1.8.7 Tutte polynomial evaluations 


Many important invariants of a matroid are generalized Tutte-Grothendieck invari- 
ants, and hence are evaluations of the Tutte polynomial. In fact, many results outside 
of matroid theory fit naturally into this framework. In this section we collect, without 
proofs, results of this sort in many different areas of mathematics and applications. 

One can probably prove every statement in this section by proving that the quan- 
tities in question satisfy a deletion—contraction recursion; many of the results also 
have more interesting and enlightening explanations. The wonderful surveys [47], 
[75], and [212] include most of the results mentioned here; we provide references for 
the other ones. 
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1.8.7.1 General evaluations 
Let M be any matroid of rank r. 
e The number of independent sets is T(2, 1). 
e The number of spanning sets is T(1,2). 
e The number of bases is T(1,1). 
e The number of elements of the ground set is log, T (2,2). 
e The Mobius number is 1(M) = uz, (0,1) = (—1)"7(1,0). 
e We have x(q) = (—1)"”T (1 — q,0) for the characteristic polynomial: 


au(q)= Y wO,F)q"™). 
FeLy 


(Note that if is a hyperplane arrangement of rank r in k@ and M is its 
matroid, then the characteristic polynomial 7,7 (q) defined in Section 1.7.2 is 


given by ¥./(q) = q*-"4u(Q)-) 


e The beta invariant of M is defined to be B(M) = [x!y°]T(x,y) = 
[x°y']T (x,y). It has some useful properties. We always have B(M) > 0. We 
have B(M) = 0 if and only if M is disconnected. We have B(M) = 1 if and 
only if M is the graphical matroid of a series-parallel graph; that is, a graph 
obtained from a single edge by repeatedly applying series extensions (con- 
vert an edge uv into two edges uw and wv for a new vertex w) and parallel 
extensions (convert one edge uv into two edges joining u and v). There are 
similar characterizations of the matroids with B(M) < 4. 


The independence complex of ™ is the simplicial complex consisting of the 
independent sets of M. Recall that the f-vector (fo,...,f-) and h-vector (ho,...,h,) 
are defined so that f; is the number of sets of size i and Yhjx"~! = Y fi(x — 1)". 
Then the following hold. 


e The reduced Euler characteristic of the independence complex is T(0,1). 
e The f-polynomial of the independence complex is Y;; fix! = x"T (1 + i, 1). 
e The h-polynomial of the independence complex is jx! = x"T ( i, 1). 


This polynomial ¥,;h;x"~! = T(x, 1) is also known as the shelling polynomial of M. 

There are many other interesting geometric/topological objects associated 
to a matroid; we mention two. Bjérner [37] described the order complex 
of the (proper part of the) lattice of flats A(Ly — {0,1}), showing it 
is a wedge of T(1,0) spheres. The Bergman fan Trop(M) = {w © R® : 
for every circuit C, maxccc WwW, is achieved at least twice} is the tropical geometric 
analog of a linear space. The Bergman complex is its intersection with the hyper- 
plane )';x; = 0 and the unit sphere yx? = |. Ardila and Klivans showed A(M) and 
A(Ly — {0, 1}) are homeomorphic. [14] Then the following hold. 
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e [37] The reduced Euler characteristic of the order complex A(Ly — {0, 1}) is 
T(1,0). 


e [14] The reduced Euler characteristic of the Bergman complex 4(M) is 
T(1,0). 


e [224] The Poincaré polynomial of the tropical cohomology of Trop(M) is 
qT(1+1/q,0). 


One of the most intriguing conjectures in matroid theory is Stanley’s h-vector 
conjecture [186], which states that for any matroid M, there exists a set X of mono- 
mials such that 


e if mand m/’ are monomials such that m € X and m'|m, then m’ € X, 
e all the maximal monomials in X have the same degree, 
e there are exactly h; monomials of degree iin X. 


This conjecture has been proved, using rather different methods, for several families: 
duals of graphic matroids, [140], lattice path matroids [176] cotransversal matroids 
[151], paving matroids [141], and matroids up to rank 4 or corank 2 [62, 117]. The 
general case remains open. 


1.8.7.2 Graphs 


Let G = (V,E) be a graph with v vertices, e edges, and c connected components. Let 
M(G) be the matroid of G and T (x,y) be its Tutte polynomial. 


Colorings. Recall that the chromatic polynomial y7¢(q) of a graph, when evalu- 
ated at a positive integer g, counts the proper colorings of G with q colors. 


e We have 
Xa(q) =(-1)" “¢ T(1—4@,0). 


e More generally, for every g-coloring 7 of the vertices of G, let h() be the 
number of improperly colored edges, that is, those whose endpoints have the 
same color. Then 


th) = (t—1)"~g T (=) (1.25) 
t—1 
“Vv >I[q) 


Flows. Fix an orientation of the edges of G and a finite Abelian group H of t el- 
ements, such as Z/tZ, for some t € N. An H-flow is an assignment f : E > H of 
a “flow” (an element of H) to each edge such that at every vertex, the total inflow 
equals the outflow as elements of H. We say f is nowhere zero if f(e) 4 0 for all 
edges e. 
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e The number of nowhere zero t-flows is given by the flow polynomial 


xG(t) = (-1)? "°F (0,1-2). 


In particular, this number is independent of the orientation of G. It also does 
not depend on the particular group H, but only on its size. 


e More generally, for every H-flow f on the edges of G, let h(v) be the number 
of edges having flow equal to 0. Then 


h(f) e-v+c grr=1 
t =(t-1 T | t, ——— }. 


f:E>H 
Acyclic orientations. An acyclic orientation of G is an orientation of the edges 
that creates no directed cycles. 


e The number of acyclic orientations is T (2,0). 


e Asource (respectively, a sink) of an orientation is a vertex with no incoming 
(respectively, outgoing) edges. For any fixed vertex w, the number of acyclic 
orientations whose unique source is w is (—1)” “u(M(G)) = T (1,0). In par- 
ticular, it does not depend on the choice of w. 


e For any edge e = uv, the number of acyclic orientations whose unique source 
is u and whose unique sink is v is the beta invariant B (M(G)). In particular, 
it is independent of the choice of e. 


Totally cyclic orientations. A totally cyclic orientation of G is an orientation of 
the edges such that every edge is contained in a cycle. If G is connected, this is the 
same as requiring that there are directed paths in both directions between any two 
vertices. 


e The number of totally cyclic orientations is T (0,2). 


e Given an orientation o of G, the outdegree of a vertex is the number of outgo- 
ing edges, and the outdegree sequence of o is (outdeg(v) : v € V). The num- 
ber of distinct outdegree sequences among the orientations of G is T(2,1). 


e For any edge e, the number of totally cyclic orientations of G such that every 
directed cycle contains e is equal to 2B(M(G)). In particular, it is indepen- 
dent of e. 


Eulerian orientations. Recall that an orientation of G is Eulerian if each vertex 
has the same number of incoming and outgoing edges. Say G is 4-regular if every 
vertex has degree 4. 


e A 4-regular graph G has (—1)"*°T¢(0, —2) Eulerian orientations or ice con- 
figurations . (They are in easy bijection with the nowhere-zero Z3-flows.) 
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Chip firing. Let G = (V,£) be a graph and let g € V be a vertex called the bank. 
A chip configuration is a map 0: V > Z with 0(v) > 0 for v #q and 0(q) = 
—Y\4q 9(v). We think of vertex v as having @(v) chips. We say v #q is ready if 
O(v) > deg(v), and q is ready if no other vertex is ready. In each step of the chip 
firing game, a ready vertex v gives away deg(v) chips, giving e chips to w if there are 
e edges connecting v and w. We say a configuration @ is stable if the only vertex that 
is ready is g. We say @ is recurrent if it is possible to fire a (non-empty) sequence of 
vertices subsequently, and return to @. We say @ is critical if it is stable and recurrent. 
One can check that for any starting chip configuration, the chip firing game leads us 
to a unique critical configuration. The following result is the key to Merino’s proof 
of Stanley’s h-vector conjecture in the special case of cographic matroids: 


e [140] The generating function for critical configurations is 


y y 99) = yFl-e8() 7,(1 yy), 


6 critical 


Chip-firing games have recently gained prominence as they are intimately con- 
nected to the study of divisors on tropical algebraic curves. [24, 84] 


Plane graphs and Eulerian partitions. Let G be a connected plane graph. The 
medial graph H has a vertex on each edge of G, and two vertices of H are connected 
if the corresponding edges of G are neighbors in some face of G. This construction 
is illustrated in Figure 1.41. 


Figure 1.41 
A connected graph G and its medial graph H in bold. 


An Eulerian partition 2 of H is a partition of the edges into closed paths. A 
closed path may revisit vertices, but it may not revisit edges; and different starting 
points and orientations are considered to be the same path. Let y(7) be the number 
of paths in 7. If the edges of a vertex v of H are e1,e2,e3,e4 in clockwise order, say 
7 has a crossing at v if one of its paths uses edges e; and e3 consecutively. 


e [128] We have Tg(—1,—1) = (—1)*(—2))-!, where z, is the unique fully 
crossing Eulerian partition of H with a crossing at every vertex. 
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e [128] If II,,(H) is the set of Eulerian partitions of H with no crossings, then 


Telea)= @—1yre". 
PEM nc (H) 


In the example above, note that the fully crossing Eulerian partition of 
G consists of one Eulerian walk, in agreement with Tg(—1,—-1) =1 = 
(—1)®(—2)!"!. We have Tg(x,x) = 3x4 + 2x3. 


1.8.7.3. Hyperplane arrangements 


Let / be an arrangement of n hyperplanes in k? of rank r. Let V(.o/) be the com- 
plement of ./ in k¢. We restate some theorems from Section 1.7.2 and present new 
ones. 


e If k=R, the number of regions of V(#) is T (2,0). 
e Ifk=C, the Poincaré polynomial of V(.) is q’T (1+ aie 
e Ifk=F,, the number of elements of V(.%) is (—1)’q"-"T(1 —q,0). 


e Suppose k = R, and consider an affine hyperplane H that is in general posi- 
tion with respect to ./. Then the number of regions of ./ that have a bounded 
(and non-empty) intersection with H is (—1)"|u(M)| =T7 (1,0). In particular, 
it is independent of H. 


e Suppose k = R, and add to o an affine hyperplane H’ that is a translation of 
H € &. The number of bounded regions of UH" is B(M). In particular, it 
is independent of H. 


1.8.7.4 Algebras from arrangements 


There are several natural algebraic spaces related to the Tutte polynomial arising in 
commutative algebra, hyperplane arrangements, box splines, and index theory; we 
discuss a few. For each hyperplane H in a hyperplane arrangement ./ in k“ let /y be 
a linear function such that H is given by the equation Jy (x) = 0. 


e [208] Let Co = span{]]yeglu : BC &}. This is a subspace of a poly- 
nomial ring in d variables, graded by degree. Its dimension is T (2, 1) and its 
Hilbert series is 


3 : j n—r 1 
Hilb(Cy.9:g) = )) dim(Cyvo)jq/ =q" 'T (: +4, -) 


j20 


e [15, 57, 103, 159, 160] More generally, let C,y, be the vector space of poly- 
nomial functions such that the restriction of f to any line / has degree at most 
Pw(h) +k, where p.7(h) is the number of hyperplanes of ./ not containing 
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h. It is not obvious, but true, that this definition of C,y9 matches the one 
above. Then 


1 1 
) , Hilb(Cy,-2;q) =q" 'T (0. ~) 


Hilb(Cyv_13q) =q" 'T (1 F 


and similar formulas hold for any k > —2. 


e [45, 162, 200] Let R(.2) be the vector space of rational functions whose 
poles are in ./. It may be described as the k-algebra generated by the rational 
functions {1/ly : H € &}; we grade it so that deg(1/J7) = 1. Then 


d 


Hilb(R(.</);q) = ine (2.9). 


1.8.7.5 Error-correcting codes 


Suppose we wish to transmit a message over a noisy channel. We might then encode 
our message in a redundant way, so that we can correct small errors introduced during 
transmission. An error-correcting code is a set C C A” of codewords of length 
n over an alphabet A. The sender encodes each word into a redundant codeword, 
which is transmitted. If the channel is not too noisy and the codewords in C are 
sufficiently different from each other, the recipient will succeed in recovering the 
original message. Of course it is useful to have many codewords available under 
these constraints. 

A common kind of error-correcting code is a linear code C, which is a k- 
dimensional vector space F{. The codewords have length n and alphabet F,. The 
support of a word is the set of non-zero entries. The distance d(u,v) between 
two words u and v is the number of coordinates where u and v differ. Since 
u—v€ @, we have d(u,v) = |supp(u — v)|. The minimum distance d in the code is 
d = mincec |supp(c)|. The code C is said to be of type [n,k, d]. 


e The weight enumerator of a linear code over Fy is Wc(4,Z) = Yeec 28. 
Curtis Greene [92] discovered that it can be expressed in terms of the Tutte 
polynomial; in fact, Theorem 1.8.14 is equivalent to the equation 


tt) 


Welqut)=(1~ayte# re (HEU, * (1.26) 


The dual code of C is C+ = {x € FY : (x,c) =0 for all e € C}. An important 
example is the Hamming code H. Let /(2,n) be the rowspace of the n x 
(2” — 1) matrix whose columns are all the nonzero vectors in F5, and let 
H = &f(2,n)+. This is a largest possible code (consisting of 2?"-"—1 words) 
of length 2” — | and distance 3. 
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Florence MacWilliams [134] gave the following remarkable identity relating 
the weight enumerators of C and C+: 


We. (4,2) = eT (4, aeare : 


In light of (1.26), this is equivalent to Tutte polynomial duality: T.. (x,y) = 
Tc(y,x). It is easy to show that the weight enumerator of .2/(2,n) is 1+ (q"— 
Vz. [92, Example 3.4] Mac Williams’s Identity then gives us the weight 
enumerator of the Hamming code. 


Another nice result [107] is that if C is a linear code in F5 then Tc¢(—1,—1) = 
(-1)"|CNC+}. 


1.8.7.6 Probability and statistical mechanics 


The Tutte polynomial arises in several standard processes in probability and statisti- 
cal physics. We consider three related models. [47, 178, 212] 


Suppose each edge of a connected graph G is white with probability p and 
black with probability 1 — p (for fixed 0 < p < 1), independently of the other 
edges. The probability that the white graph is connected is given by the reli- 
ability polynomial 


Ro(p)=  Y pl —p)lF4l = (1p) pT (1 =) 


ACE | 
A spanning 


The random cluster model depends on parameters 0 < p < 1 and q > 0. 
Now we choose a white set of edges at random, and the probability of choos- 
ing A C E is p'4|(1 — p)!2 41g) /Z, where c(A) = |V| — r(A) is the number 
of components of A, and Z is a scaling constant. To know the probability of 
a particular state, it is fundamental to know the scaling constant Z, which is 
the partition function 


Z(p.q) = Y pal—pylF lg 
ACE 


: : 1- 1 
= pad — p)? **¢¢ Tg (1 A q( P) ) ; 
p “l=p 


In the g-state Potts model, a graph G = (V,E) models a set V of “atoms” 
and a set E of “bonds” between them. (When q = 2, this is the Ising model.) 
Each atom can exist in one of q states or “spins.” Each edge e = uv has an as- 
sociated interaction energy J, between u and v. The energy (or Hamiltonian) 
H of a configuration is the sum of —J, over all edges e whose vertices have 
the same spin. (The case J, > 0 is called ferromagnetic, as it favors adjacent 
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1.8.7.7 


spins being equal. The case J, < 0 is antiferromagnetic. Here we are assum- 
ing that there is no external magnetic field. If there were such a field, it would 
contribute an additional term to the Hamiltonian, and the direct connection 
with the Tutte polynomial is no longer valid.) A configuration of energy H 
has Boltzmann weight e B# where B =1/kT > 0, T is the temperature, and 
k is Boltzmann’s constant. 


The partition function is the sum of the Boltzmann weights of all configu- 
rations: 


zoawy= YT A= YT (two 


o:V> e=ijeE e=ijEE 
1 (200) Prone) 
where we = ePJe — 1. Tf all Jes are equal to J and w = es 1, then (1.25) 
gives 


Zo(q,w) =w’ °q°T (2+1,w+1). 


For general J,, a simple computation shows that 


q- "ZG q,W =a “ T] we =Zo(a.w 
ACE ecA 


where Zg(g,w) is the multivariate Tutte polynomial of Section 1.8.9. 


Other applications 


[108, 212] A knot is an embedding of a circle in R*. Itis a difficult, important 
question to determine whether a knot can be deformed smoothly (without 
cutting or crossing segments) to obtain another knot; or even to determine 
whether a given knot is actually knotted or not. Let O be the unknotted circle 
in R?. 

One common approach is to construct a function f(K) of a knot K that does 
not change under smooth deformation. If f(K) 4 f(O), then K is actually 
knotted. One such function is the Jones polynomial V (XK). If K is an alter- 
nating knot, there is a graph G associated to it such that the Jones polynomial 
of K is an evaluation of the Tutte polynomial of G. 


[164] There is a more symmetric finite field interpretation for the Tutte poly- 
nomial. Let M be an integer matrix, and let p and q be prime powers such 
that the matroid of M does not change when M is considered as a matrix over 
F,, or over F,. Then 


Tu(t—p,1—g)=(-1) (eyo, 


xerow(M),ycker(M) 
supp(x)Lisupp(y)=E 


where the rowspace row(M) is considered as a subspace of IF, and the kernel 
ker(M) is considered as a subspace of F”, and LI denotes a disjoint union. 
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[4, 41] The Tutte polynomial of the Catalan matroid is 


Tc, (x,y) — y xP) PP) 
P Dyck 


where a(P) is the number of upsteps before the first downstep of P, and b(P) 
is the number of times that P returns to the x-axis. Since C,, is self-dual, this 
polynomial is symmetric in x and y. 


In fact, the coefficient of x/y/ in Tc,, (x,y) depends only on i+ j. It would be 
interesting to find other families of matroids with this unusual property. 


[41] More generally, the Tutte polynomial of a lattice path matroid is the sum 
over the bases P of x*(”)y>(?), where a(P) and b(P) are the number of returns 
to the upper and lower boundary paths, respectively. 


[121] A T-tetromino is a T-shape made of four unit squares. Figure 1.42 
shows a T-tetromino tiling of an 8 x 8 square. The number of T-tetromino 
tilings of a 4m x 4n rectangle equals 27;,, (3,3), where Lm» is the m xn 
grid graph. This result extends to T-tetromino tilings of many other shapes 
and, more generally, to coverings of many graphs with copies of the “claw” 
graph Kj 3. 


Figure 1.42 
A tiling of an 8 x 8 rectangle by T-tetrominoes. 


1.8.8 Computing the Tutte polynomial 


As we already mentioned, computing the Tutte polynomial of an arbitrary graph or 
matroid is not a tractable problem. In the language of complexity theory, this is a #P- 
complete problem. [212] However, the results of Section 1.8.6 allow us to compute 
the Tutte polynomial of some matroids of interest. We now survey some of the most 
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interesting examples; see [142] for others. Some of these formulas are best expressed 
in terms of the coboundary polynomial 


xX+Y-1 


Xo (X,Y) = (y—1)!™ Ty (x,y), where x = a aan ak’ 


y=yY. 


Almost all of them are most easily proved using the Finite Field Method (Theorem 
1.8.14) or its graph version (1.25). 


e For the uniform matroid Ug, we have Ty,,,(x,y) = Li-y Cra + 
n—r (n—j-l\,j 
Liat ( r-1 yl. 


e If M“) is the matroid obtained from M by replacing each element by k copies 
of itself, then 


k—1 k-2 
Batt oes ANB icie As aed 
Typ (sy) = ty + ty 41)'T Cat eet). 


This formula is straightforward in terms of coboundary polynomials: 
Xu! (X,Y) =Xy (XY 3 ). For an extensive generalization, see Section 1.8.9. 


e [6, 145] Root systems are arguably the most important vector configura- 
tions; these highly symmetric arrangements play a fundamental role in many 
branches of mathematics. For the general definition and properties, see for 
example [106]; we focus on the four infinite families of classical root sys- 


tems: 
An-1 = {ei—ej,: 1<i<j<n} 
Bn = {ei—ej,e;+ej:1<i<j<nbUf{e;:1<i<n} 
C, = {er—ej,e;tej:1<i< j<n}U{2e:1<i<n} 
Dn = {ei—ej,e;+ej:1<i<j<n}. 


See Figure 1.43 for an illustration. 


Figure 1.43 
The root systems Az, Bz,C2, and D2, respectively. 
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Let the deformed exponential function be F(a@,f) = ¥,+9(0" B2)) /n!. 
Then the Tutte generating functions of the infinite families A and ® = 
B,C,D.: 
Z Z 
T4(X,Y,Z) =1+X Ye Xa, oY), Te(X,Y,Z) = VX, (X oo ee, 
n>l j n>0 


are given by 


Ty = F(Z,Y)*, 

Tg = F(2Z,Y)*-V?PF(yz,yY?), 
To = F(2Z,Y)%-)/?F(¥Z,Y"), 
Tp = F(2Z,Y)*-YPF(z,Y?). 


Aside from the four infinite families, there are a small number of exceptional 
root systems, which are also very interesting objects. Their Tutte polynomials 
are computed in [61]. 


Since the matroid of the complete graph K,, is isomorphic to the matroid of 
the vector configuration A,_;, the first formula above is a formula for the 
Tutte polynomials of the complete graphs, proved originally by Tutte [202]. 


Similarly, the coboundary polynomials of the complete graphs K,, , are given 
by 
zn ZH zn za\* 
14+X XK, XY) = ym) 
», ; m! n! ey m! n! 


(m,n) (0,0) 


[27, 145]. The Tutte polynomial of the arrangement ©/(p,n) of all linear 
hyperplanes in F’, is best expressed in terms of a “p-exponential generating 
function”: 


u" (U3 Poo Lippetpt-l ou" 
Kilo Ao") = yleete ep = 
pe a (P:P)n (Xu P)oo Xu (P3P)n 


where we define (a;p)o. = (1 — a)(1 — pa)(1 — p’a)--» and (a;p)n = 
(1—a)(1—pa)---(1—p"™""a). 


Equation (1.25) expresses the Tutte polynomial of a graph G as the number 
of g-colorings of G, weighted by the number of improperly colored edges. 
When G has a path-like or cycle-like structure, one may use the transfer- 
matrix method of Section 1.4.1.2 to carry out that enumeration. This was 
done for the grid graphs L,, for small fixed m [48] and for the wheel graphs 
W, [51] shown in Figure 1.44. For example, the Tutte polynomial of the 
wheel graph is 


Tw, (x,¥) = an (6+ VP 4a) + a (6 — VP =4a)' +a—b 


where b = ere and ad =xy. 
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Figure 1.44 
The grid graph L4.3 and the wheel graph Ws. 


1.8.9 Generalizations of the Tutte polynomial 


The multivariate Tutte polynomial. The multivariate Tutte polynomial of a ma- 


troid M is 
w) = y oo [| 
ACE ecA 


where qg and (we)ece are indeterminates. When all w. = w, we get Z(g,w) = 
wT (£+1,w+1). 

Notice that Zu(q:w) determines E and r, and hence it determines M. In fact, 
among the many definitions of a matroid, we could define M to be Zu(q:w). This is 
a useful encoding of the matroid. 


e We saw in Section 1.8.7 that for a graph G and a positive integer g, the multi- 
variate Tutte polynomial is equal to the partition function of the g-state Potts 
model on G. 


e For a vector a € N”, let M(a) be the matroid M where each element e is 
replaced by a- copies of e. It is natural to ask for the Tutte polynomials of 
the various matroids M(a). For each a, 


Tyg I= = Za (EAP) Fe sd) 


The generating function for the Tutte polynomials of all the matroids M(a) 
turns out to be equivalent to the multivariate Tutte polynomial, disguised 
under a change of variable: 


= et (= Do - SM... a), 


l—-yw, >” 1-ywy 


e Foran algebraic interpretation of the multivariate Tutte polynomial, see [15]. 
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The arithmetic Tutte polynomial. When we have a collection A C Z” of integer 
vectors, there is a variant of the Tutte polynomial that is quite useful. The arithmetic 
Tutte polynomial is 


=F m(B)(x— 1)" (y — 181-7) 
BCA 


where, for each B C A, the multiplicity m(B) is the index of ZB as a sublattice of 
(span B) MZ”. Using the vectors in B as the columns of a matrix, m/(B) is the greatest 
common divisor of the minors of full rank. This polynomial is related to the zonotope 
of A [190, 56]: 


e The volume of the zonotope Z(A) is M4(1, 1). 


e The zonotope Z(A) contains M4 (2, 1) lattice points, M4(0, 1) of which are in 
its interior. 


e The Ehrhart polynomial of the zonotope Z(A) is g’M(1 -, ,1). 


Let T = Hom(Z",G) be the group of homomorphisms from Z” to a multiplicative 
group G, such as the unit circle S! or F* = F\{0} for a field F. Each element a € A 
determines a (hyper)torus T, = {t € T : t(a) = 1} in T. For instance a = (2, —3,5) 
gives the torus xy~3z> = 1. Let 


F(A)={%:aeA}, R(A=T\ VU T 
ac TF (A) 


be the toric arrangement of A and its complement, respectively. 


e [72, 144] If G=S!, the number of regions of R(A) in the torus (S!)’ is 
Ma(1,0). 


e [59, 144] If G = C*, the Poincaré polynomial of R(A) is g’M4(2 + 1,0). 


e (43, 11] IfG= Les 4, Where q+ 1 is a prime power, the number of elements 
of R(A) is (—1)"q"-"M, (1 —q,0). Furthermore, 


YH) = (0-1 "My (a) 


peFiy 
where h(p) is the number of tori of .7(A) that p lies on. 


As with ordinary Tutte polynomials, this last result may be used as a finite field 
method to compute arithmetic Tutte polynomials for some vector configurations A. 
However, at the moment there are very few results along these lines. One exception 
is the case of root systems; the study of their geometric properties motivates much of 
the theory of arithmetic Tutte polynomials. Explicit formulas for the arithmetic Tutte 
polynomials of the classical root systems A,,B,,C,, and D, are given in [11], based 
on the computation of Example 15 in Section 1.3.3. 
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1.8.10 Matroid subdivisions, valuations, and the Derksen-Fink 
invariant 


A very interesting recent development in matroid theory has been the study of ma- 
troid subdivisions and valuative matroid invariants. We offer a brief account of some 
key results and some pointers to the relevant bibliography. 

A matroid subdivision is a polyhedral subdivision Y of a matroid polytope 
Py where every polytope P € # is itself a matroid polytope. Equivalently, it is a 
subdivision of Py whose only edges are the edges of Py. See Figure 1.45 for an 
illustration. In the most important case, M is the uniform matroid Uy, and Py is the 
hypersimplex A(d,n). 


S- PBB 


Figure 1.45 
A matroid subdivision of A(2,4). 


Matroid subdivisions arose in algebraic geometry [94, 113, 126], in the theory of 
valuated matroids [70, 147], and in tropical geometry [180]. For instance, Lafforgue 
showed that if a matroid polytope Py has no nontrivial matroid subdivisions, then 
the matroid M has (up to trivial transformations) only finitely many realizations over 
a fixed field F. This is one of very few results about realizability of matroids over 
arbitrary fields. 

The connection with tropical geometry is a rich source of examples. Let K be the 
field of Puiseux series agxt!N + qyxlmtD)/N 4 -++ (where ag,a1,...€ C,m,N € Z and 
N > 0). Roughly speaking, every subspace L of K” may be tropicalized, and there 
is a canonical way of decomposing the resulting tropical linear space into Bergman 
fans of various matroids. These matroids give a matroid subdivision of A(d,n). For 
some nice choices of L, the corresponding matroid subdivisions can be described 
explicitly. For details, see for example [14, 166, 180]. 

There is also a useful connection with the subdivisions of the product of sim- 
plices Ag_1 < An—¢—1, which are much better understood, as we discussed in Section 
1.6.4. Note that the vertex figure of any vertex of A(d,n) is Ag_) x Ay—a—1. Ev- 
ery matroid subdivision of A(d,n) then induces a “local” polyhedral subdivision of 
Ag—1 X An—a—1 at each one of its vertices. Conversely, we can “cone” any subdivision 
S of Ag_; X Aj—ga_) to get a matroid subdivision that looks like .Y at a given vertex. 
[98, 166] 
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Enumerative aspects of matroid subdivisions. Say a face P of a subdivision W 
of Py is internal if itis not on the boundary of Py. Let P™ be the set of internal faces 
of Y. Currently, the most interesting enumerative question on matroid subdivisions 
is the following. 


Conjecture 1.8.15 (Speyer’s f-vector conjecture [180]) A matroid subdivision of 


the hypersimplex A(d,n) has at most aa interior faces of dimension 


n—c, with equality if and only if all facets correspond to series-parallel matroids. 


Speyer constructed a subdivision simultaneously achieving the conjectural max- 
imum number of interior faces for all c. In attempting to prove this conjecture, he 
pioneered the study of valuative matroid invariants. We say that a matroid invariant 
f is valuative if for any matroid subdivision Y of any matroid polytope Py we have 


f(M)= Yo (-1jameeran tn £(Mi). 


Pu; E Pint 


There are obvious matroid valuations, such as the volume or (thanks to Ehrhart 
reciprocity) the Ehrhart polynomial of Py. Much more remarkably, we have the fol- 
lowing result. 


Theorem 1.8.16 [180] The Tutte polynomial Ty(x,y) is a matroid valuation. 


Corollary 1.8.17 The f-vector conjecture is true for c = 1. 


Proof. By Theorem 1.8.16 the beta invariant B(M) = [x!y°]Tiy(x,y) is also a ma- 
troid valuation. Recall that B(N) = 0 if and only if N is not connected (or, equiva- 
lently, if Py is not a facet of A), B(N) = 1 if and only if N is series-parallel, and 
B(N) > 2 otherwise. Also B(Uan) = (‘7 a Therefore, in a matroid subdivision of 
A(d,n) we have 


—2 
rs B(Uan) = = a B(N) > (number of facets of Y) 
d—1 Pyed 
EP 
P facet 
with equality if and only if every facet is a series-parallel matroid. | 


For matroids M realizable over some field of characteristic 0, Speyer [181] used 
the K-theory of the Grassmannian to construct another polynomial invariant gy(t), 
which he used to prove the f-vector conjecture for matroid subdivisions whose ma- 
troids are realizable in characteristic 0. 


Theorem 1.8.18 [181] The f-vector conjecture is true for matroid subdivisions con- 
sisting of matroids that are representable over a field of characteristic 0. 


Speyer’s proof of this result relies on the nonnegativity of gjy(t), which is proved 
geometrically for matroids realizable in characteristic 0. The nonnegativity of gy(t) 
for all M, which would prove Conjecture 1.8.15 in full generality, remains open. 
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The Derksen-Fink invariant. Many other natural matroid functions were later dis- 
covered to be valuative. [13, 35, 64]. An example of a very general valuation on ma- 
troid polytopes from [13] is the formal sum R(M) = Yace Ra y(a) Of symbols of the 
form Rs; where S is a subset of EF and k is an integer. Many other nice valuations 
can be obtained from this one. In fact, the matroid M can clearly be recovered from 
R(M). (However, note that R is not a matroid isomorphism invariant.) 

Eventually, Derksen [64] constructed a valuative matroid invariant, which he and 
Fink proved to be universal. [65] We call it the Derksen—Fink invariant. They gave 
several versions; we present one that is particularly simple to define. 

Let Matroids,,, be the set of matroids on [n] of rank r. Let @, be the set of n! 
complete chains .Y :@ = So © S} © +++ © Sn—-1 | Sy = [n]. For a complete chain .Y 
and a matroid M = ((n],r), let r[S;_1,5;] =r(S;) —r(S;-1) for 1 <i <n; this is always 
0 or 1. The Derksen—Fink invariant of M is 


G(M) = YP Uerig81]r181Sae-stlSn-15) 
SCCn 


in the vector space U spanned by the (") formal symbols Uy, .....,, where w1,...,Wn € 
{0,1} and wy +---+Wa =r. 


Theorem 1.8.19 [65] The Derksen—Fink invariant Y : Matroids,, — U is a uni- 
versal valuative matroid invariant; that is, for any valuative matroid invariant 
f : Matroidsy,, — V there is a linear map 7 :U — V such that f = 10. 
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2.1 Introduction 


This chapter should have really been written by Philippe Flajolet (1948-2011), who 
could not make it this time. He coined the name analytic combinatorics. The present 
author knew Flajolet since 1979 and followed all the developments closely since 
then, also being a coauthor on various occasions. Flajolet and his followers started 
out in analysis of algorithms, a subject founded by Knuth in his series of books 
The art of computer programming; it became clear over the years that many tech- 
niques from classical mathematics had to be unearthed and many new ones had to 
be discovered. Flajolet was a pioneer in this direction; Doron Zeilberger called him 
a combinatorialist who became an analyst, and it was his understanding that combi- 
natorics should have an analytic component, like number theory has analytic number 
theory. Flajolet, apart from being an exceptional problem solver, had a strong desire 
to be clear and systematic. Eventually, with coauthor Robert Sedgewick, after many 
years of preparations, the book Analytic Combinatorics [24] was published. It has 
810 pages, and only a fraction of it can be represented here. 

When analyzing algorithms, there is often an algebraic (combinatorial) part, fol- 
lowed by an asymptotic (analytic) part. And, indeed, analytic combinatorics, as un- 
derstood by Flajolet, follows the same pattern. The central objects are generating 
functions. First, through combinatorial constructions (a bit reminiscent of grammars 
for formal languages), symbolic equations for the objects are obtained. There are 
rules for how to translate the symbolic equations into equations for the associated 
generating functions. From here, there are a few typical scenarios. Ideally, one can 
write explicit expressions for the generating functions, and then get from them ex- 
plicit expressions for the coefficients, which are the numbers of interest. But often, 
these expressions are involved, and one needs asymptotic techniques. It is often better 
to derive asymptotic equivalents directly from the generating functions. Sometimes, 
one cannot solve equations and thus has no explicit form for the generating functions. 
But even then there might still be hope. Asymptotic methods (in combinatorics) have 
been known for decades, and we might cite De Bruijn’s book [5] and Odlyzko’s trea- 
tise [40]. One of Flajolet’s favorite methods was the Mellin transform. He wrote a 
series of survey papers about it, and a draft of some 100+ pages about it existed, but 
eventually did not make it into the book [24]. 

We follow these guidelines and include material here that, in one way or another, 
can be traced back to [24]. Whatever we cite from this great book we do with due 
respect. Even in places where it is not explicitly said, the concept and notations are 
probably taken from this authoritative text, since we firmly believe that it is vain to 
try to improve on the masters themselves. 

The choice is perhaps a bit personal; during some 35 years as a working math- 
ematician, I came across many things that are useful and also necessary to know. 
Whatever is included here, was interesting to myself over the years. I hope that the 
selection presented here will be useful for the readers of this handbook as well. 
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Plan of this chapter. We start with combinatorial constructions, for both unla- 
beled and labeled objects. On the way, we discuss various classical combinatorial 
objects, like compositions, partitions, trees, set partitions, etc. Then we elaborate 
on techniques related to generating functions. Before we move to asymptotic con- 
siderations, we need a few preparations, like the Gamma function, the zeta func- 
tion, the harmonic numbers, etc. Then we move to the Mellin transform. Here, we 
start with so-called harmonic sums, move to digital sums, and eventually to divide- 
and-conquer type recursions. Important here is the Mellin inversion formula and the 
Mellin-Perron summation formula. Then we discuss Rice’s method, which is based 
on the Cauchy integral formula, and a running example that is both instructive and 
important: approximate counting. Then we sketch singularity analysis of generating 
functions, which is a toolkit allowing us to translate from the local behavior of the 
generating functions around their (dominant) singularities to the asymptotic behavior 
of its (Taylor-)coefficients. Another interesting running example is the one of longest 
runs in strings consisting of two symbols. Another important asymptotic technique, 
the saddle point method, is only sketched. The last section deals with Gaussian lim- 
iting distributions and how they can be obtained in an important family of special 
instances; these developments are due to H.-K. Hwang. This section must be seen 
as an appetizer; it will increase the desire to read more comprehensive texts about 
limiting distributions in combinatorial analysis. 

Let us collect a few conventions and notations and facts that will be encountered 
later: 


Iverson’s notation. We write [P] = 1 when condition P is true, [P] = 0 otherwise. 
This notation is more flexible, say, than Kronecker’s 6;,; = [i = j]. 


Probability generating functions and moments. Assume that the power series 
f(z) has non-negative coefficients and f(1) = 1 (a probability generating function). 
Then expectation and variance (when they exist) are given by 


2 


f=f'(), Va f"()+f0)- (FO). 


Coefficient extraction of power series. If f(z) = Y,59 42", then [z"| f(z) = dn. 


2.2 Combinatorial constructions and associated 
ordinary generating functions 
Let be a denumerable set with a size (a non-negative number) associated to each 


of its elements. We often write |a| for the size of a € &. Furthermore, we assume 
that, for each n, there is only a finite number of elements of size n in /; we call it 
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dn. Then we define the associated (ordinary) generating function 


A(z) = 3 ainZ”: 
n>0 
For the time being, this is just a formal construction, but later on, we will interpret 
A(z) as a function of a complex variable z. This will be particularly useful when we 
discuss how to get asymptotic equivalents for the numbers a,. At the moment, we are 
just interested in combinatorial constructions, and how they are reflected by their as- 
sociated generating functions. Such constructions allow us to start from basic objects 
and create more complicated ones. Another, but equivalent point of view is decom- 
position, where a certain combinatorial class of objects is (uniquely) decomposed 
into simpler ingredients. Combinatorial constructions are also relevant in symbolic 
algebra systems. 
The most common constructions are now discussed. 


Union. Let # and @ be such combinatorial classes with associated generating 
functions B(z) and C(z) and assume that the classes are mutually disjoint. Then the 
(disjoint) union. = B+@, also written as o = BUG, has associated generating 
function A(z) = B(z) +C(z). This follows from the elementary ay, = by + Cn. 


Product. Let & and @ be such combinatorial classes with associated generating 
functions B(z) and C(z). Then we form the (Cartesian) product # = & x @. The size 
of an object (b,c) is defined to be |b| +|c|. Then the associated generating function 
is A(z) = B(z)C(z). This follows from the fact that 


n 
an = y DkCn—k; 
k=0 


and this is just the Cauchy product of two series: 


B(z)C(z) = y Dnz” : y Cnz" = y (3 Per) Ze = A(z). 


m>0 n>0 n>0 


This idea immediately extends to several factors, not just two. In particular, for a fixed 
number k, we can consider k-tuples (x1,...,x;), where all x; € &, and |(x1,...,x%)| = 
|x| +--+ |x,|. Then the generating function associated to o/* is A*(z). 


Sequence. Let & be a combinatorial class, that does not contain elements of size 
0. Then we form 

A=BABAB +, 
which describes sequences of elements of Z. The (unique) sequence of zero elements 
is traditionally written as €. We write </ = SEQ(#), and the associated generating 


function is ; 


~ 1B)’ 
Especially when dealing with languages (sets of words), the notion 4 instead of 
SEQ(¥) is common; then also. Zt = Z14+ BH +---. 


A(z) =1+B(z)+B?(z) ++: 
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Power set. Fora given &, we form finite sets of elements taken from Y; the result 
is @ = PSET(#), and the size of such a set is defined to be the sum of the sizes of 
its elements. We must assume that & does not contain an element of size 0. We have 


@ = (€+{Bi}) x (€+ {Bo}) x--- 


for an enumeration (1, B2,...) of the class 4. Now let 


B(z) = y Bnz", 


n>1 


then we can compute 


A(z)= [J + z/Bly — [[G+2")” 


BEB n>1 
i (—1-'e* 
= exp B,log(1 +z") ) =exp B, ), —————_ 
(35 satett+2) <0 (J a4, Uy 
__ (BQ) _ B@) , BG’) 
=exp (5 poy se), 


The operations with infinite series are justified, since, for given n, only a finite num- 
ber of B;’s contribute to An. 


Multiset. This is very similar to forming sets, but now repeated elements are al- 
lowed, leading to multisets. The computation is similar: 


A(z) =[[U 42742" 42°" +--+ )Pn 


n>1 
1 zk 
=exp (Br los; =) =exp (5. ¥ T) 
B22) B(23 
Sg 0 BE AO) 2 


Cycles. For a given &, we form cycles of elements taken from 4%, where again 
Bo = 0. A cycle is (b1,...,bn) with b; € &. It is identified with all cyclic rotations. 
So, for example, (a,b,a,b) = (b,a,b,a), but (a,a,b,b) is a different cycle. Again, 
the size of a cycle is the sum of the sizes of its elements. Then, for the associated 
generating functions, 

1 


B(zk)’ 


9 (k) 
A(z) = lo 
(z) d 7 eT 
where @(k) is Euler’s totient function; in other words, the number of i’s less than k 
that are relatively prime to k. 
The proof of this relation will not be given; it is usually done using Polya’s enu- 
meration theory (enumeration under group action, here just the cyclic group). 
Constructions under restrictions are also considered, for example SEQ;(¥), 
SEQ>;(4) of sequences with exactly k or > k elements, and various others. 


178 Handbook of Enumerative Combinatorics 


Further constructions will be introduced in this text when they occur. 
Now we turn to a few examples. 


Compositions. A composition of a positive integer n is a representation n = ij + 

-- + ix with positive integers i;; the number k is referred to as the number of parts. 
We can interpret the integers .% = {1,2,...} = {e,e0e,e0ee,...} = SEQ: {e}. The 
size of integer i ~ e! is just i, and so 


Further, compositions are described by @ = SEQ; (-%), whence 


fees Ad 2 
122° 


and [,, the number of compositions of n, is given by J, = 2"! which is also easy to 
see directly. 


Partitions. They are defined like compositions, except that the order of the terms 
(“parts”) is irrelevant. They can be seen as multisets of .7; the multiset construction 
then gives us 


P(e) =exp (12) + 4 FE) 4...), 


2 3 
This form is, however, not very useful; the more natural way to write this is 
P(r) = 1 
(z) st I] 1—z" . 
n>1 
ij +2i2+3i3+-- 


On expanding the product, a typical term is z which just describes a 
partition (7; ones, i2 twos, i3 threes, etc.) The size is the sum of the parts. There will 
be a separate section providing the very basic elements of the extremely rich and 
useful theory of partitions. 


Some families of trees. The class of binary trees # is either an external node or a 
root (an internal node) followed by a left and a right subtree, both again binary trees. 
This recursive definition can be stated as 4 = +0-4-@. External nodes are not 
counted when one speaks about size, and sometimes not drawn. The equation 


B(z) =1+zB?(z) 


is immediate, leading to 


eae or 


so that binary trees with n (internal) nodes are enumerated by Catalan numbers 
1 (2n 
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The generalization to ¢-ary trees is immediate: The root has an ordered list of t 
subtrees, and tf = 2 means binary trees. 

Another important family is Y, the family of planar trees. They are known un- 
der several names: planted plane trees, plane trees, ordered trees, ... There is a root 
node and a sequence of r planar subtrees (r > 0). Thus Y=o0+0:-Y+0:-P-P 
+0:-P-P-A-+--- Consequently 


P(z) = zP(z) +2P2(z) +2P3(z) +++ = — 
(z) = zP(z) + 2P*(z) + zP°(z) 1—P@) 
_1-vVI-&_ 1 /2n-2) , 

7 2, <8 n—1 ae 


There are two standard bijections: Between planar trees and binary trees the rotation 
correspondence. Take a planar tree, let only the leftmost edge survive, connect sib- 
lings instead, cut off the root and turn the tree by 45° to obtain a binary tree with one 
node less. This is reversible. This construction is described in more detail in many 
textbooks. Figure 2.1 shows an example. 


Figure 2.1 
A planar tree with eight nodes (seven edges) and the corresponding binary tree with 
seven internal nodes. 


Planar trees are also in bijection with non-negative lattice paths (Dyck paths), 
which are described a little later in this text. See Figure 2.2 for an example. 


Figure 2.2 
A planar tree with eight nodes (seven edges) and the corresponding Dyck path of 
length 12 (=semi-length 6). 
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As this example shows, one just walks around the tree and records the steps (up 
or down) in a diagram. 

Much more about these and other families of trees can be found in this handbook 
in various chapters that specialize on trees. 


Set partitions. A partition of the set {1,2,...,n} into k blocks consists of k 
nonempty subsets that are mutally disjoint and whose union is the full set. The num- 
ber of them is denoted by eh a Stirling subset number (in the older literature Stirling 
numbers of the second kind). In order to describe these set partitions symbolically, 
we call the smallest element in each block the leader. Then we order the blocks ac- 
cording to the block leaders in ascending order. In this way, it makes sense to talk 
about block 1, block 2, ..., block k. And now we write a string b}b2...by,, where b; is 
the number of the block in which i lies (its address). Such a string has the properties 
that b; = 1, and if a new number appears for the first time (scanning the string from 
left to right), it is one higher than the previous highest number, and altogether all 
the numbers 1,...,k appear. Example. The string 112122313241 codes the set parti- 
tion {1,2,4,8, 12}, {3,5,6, 10}, {7,9}, {11}. The set of admissible strings admits the 
following representation: 


1SEQ(1)2SEQ(1+2)3SEQ(1+2+4+3)...KSEQ(1+---+k). 


This translates into the generating function 


ee, in 
1—z1—2z °° 1—kz’ 


partial fraction decomposition gives 


MombE Carpal RE Cor 


g&) (z) 


2.3. Combinatorial constructions and associated 
exponential generating functions 


We are now discussing labeled classes. The idea is as follows. In the previous section, 
the function “size” was abstractly introduced. Now, if an object has size n, we assume 
that there are n atoms present, and we label them. Here is a precise definition. 


Definition 2.3.1 A weakly labeled object of size n is a graph whose set of vertices 
is a subset of the integers. Equivalently, we say that the vertices bear labels, with 
the implied condition that labels are distinct integers from Z. An object of size n is 
said to be well-labeled, or, simply, labeled, if it is weakly labeled and, in addition, 
its collection of labels is the complete integer interval [1..n]. A labeled class is a 
combinatorial class that consists of well-labeled objects. 
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We will also use reduction of labels: For a weakly labeled structure of size n, 
this operation reduces its labels to the standard interval [1..n] while preserving the 
relative order of labels. For instance, the sequence (7,3,9,2) reduces to (3,2,4,1). 
We use P (a) to denote this canonical reduction of the structure a. 

In order to count labeled objects, we appeal to exponential generating functions. 
The exponential generating function of a sequence (A,,) is the formal power series 


gt 
A(z) = y An “a 
n! 
n>0 

The exponential generating function of a class is the exponential generating func- 
tion of the numbers A,, = card(.%,). It is also said that the variable z marks the size 
in the generating function. With the standard notation for coefficients of series, the 
coefficient A, in an exponential generating function is then recovered by 


An = nz" A(z), 
since [z"|A(z) =A,/n!. 


Neutral and atomic classes. It proves useful to introduce a neutral (empty, null) 
object € that has size 0 and bears no label at all, and consider it as a special labeled 
object; a neutral class & is then by definition & = € and is also denoted by 1. The 
(labeled) atomic class 2 = (© is formed of a unique object of size 1 that, being 
well-labeled, bears the integer label (1). The exponential generating functions of the 
neutral class and the atomic class are, respectively, E(z) = 1, Z(z) =z. 


Labeled product. The labeled product of & and @, denoted 4x @, is obtained 
by forming ordered pairs from 4 x @ and performing all possible order-consistent 
relabelings. When W = 4x @, the corresponding counting sequences satisfy 


“(n 

An = »y (7) Bic 
k=0 

the binomial coefficients count the relabelings. The new object has size k+ (n—k) = 

n; k of the numbers {1,...,2} are selected for the labels of the first component, 

n—k for the second. But this is just the way exponential generating functions are 

multiplied: 


n n 


Z eke z 
Ans ar “ Lon. 


n>0 * n>0 


Sequences. General sequences and sequences with k factors can be formed as be- 
fore. Here and in the following we assume again that € ¢ &. 


SEQ(Z) = {E}+ B+ BrB+ BrBr B+ = |JSEG(A). 


k>0 
The translations into exponential generating functions are 
1 
A(z) = B(z)* and A(z) = , 
(2) = Bl) =a 


respectively. 
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Sets. We denote by SET;,(A) the class of k-sets formed from Z&. The set class is 
defined formally as the quotient SET,(Y4) := SEQ;(¥Y)/R, where the equivalence 
relation R identifies two sequences when the components of one are a permutation of 
the components of the other. A “set” is like a sequence, but the order between com- 
ponents is immaterial. The (labeled) set construction applied to Z, denoted SET(Y¥), 
is then defined by 
SET(B) = |) SET, (A). 
k>0 
The translations into exponential generating functions are 


respectively. 


Cycles. We start with k-cycles. The class of k-cycles, CyC;,(¥) is formally defined 
to be the quotient CYc,(#) := SET;(¥)/S, where the equivalence relation S iden- 
tifies two sequences when the components are one cyclic permutation of the compo- 
nents of each other. A cycle is like a sequence whose components can be cyclically 
shifted, so that there is now a uniform k-to-one correspondence between k-sequences 
and k-cycles. We assume that @ #@ and k > 1. Then 


A=CYC(F) = Al(zZ= £B(2), 


@=CYC(Z) = AZ)= PO =i aa 


In the sequel we describe a few important combinatorial objects as families of 
labeled objects. 


Surjections. Fix some integer r > 1 and let RY denote the class of all surjections 
from the set {1,...,2} onto {1,...,7} whose elements are called r-surjections. We set 


AY) = Un>1 RO and compute the corresponding exponential generating function, 


R”) (z). We observe that an r-surjection @ € RY is determined by the ordered r- 
tuple formed from the collection of all preimage sets, (@~'(1),@~!(2),...,67!(r)); 
they are disjoint non-empty sets of integers that cover the interval [1..n]. One has the 
combinatorial specification 


RO =SEQl(V), V=SETs1(X), 


(a surjection is a sequence of non-empty sets), from which we conclude R”) (z) = 
(e* — 1)". From this, we find also 


j 
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Set partitions into 7 blocks. Let PY) denote the number of ways of partition- 
ing the set {1,...,} into r disjoint and non-empty equivalence classes (blocks). We 


set A) = Un>1 SM): the corresponding objects are called set partitions, as defined 
earlier. The enumeration problem for set partitions is closely related to that of surjec- 
tions: 


(e-1" 


r} 


A) =Set(V), W=SETS (LF) = SM(z)= 
Thus we find again the formula for the Stirling set partition numbers: 


(rag Gors 


If we want to consider all surjections (respectively, set partitions) instead of only 
those that consist of r blocks, then we simply have to sum over r. This leads to 


1 
R = os | r— — 
@=PE-V'=5-5 
and (e—1) 
_ e—1)' = el 
S(z) => ae ee 


The numbers R, = n![z"|R(z) and S, = n![z"]S(z) are called surjection numbers and 
Bell numbers, respectively. Clearly, 


n n 
ae prt") cad Sy a 
r>0 r>0 


We have ‘ 
1 1 es 
R(z) = =) a 
21- S 5 Qi+1 
therefore 
[” 
Rn se QI+1 
I>0 
Similarly, 
1 J 
S@)=se ==, 
7 ej I! 
whence 


This is known as Dobinski’s formula. 
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The present approach is also flexible with respect to restrictions. For example, 
exp(e,(z) — 1), with the truncated exponential series 


2 rad 


=] Zz 
ep(z) = a es 


corresponds to partitions with all blocks of size < b, e® —'~? corresponds to partitions 
with no singletons, and cosh(e* — 1) to partitions with an even number of blocks. 


Restricted words and random allocation. Consider an alphabet with r letters, 


say, {a1,...,a,}. For a word of length n, the sequence {set of indices of letter a;}, 
..., {set of indices of letter a,} is forming an “ordered” partition of the sets of labels 
{1,...,n}; without restrictions, this yields the exponential generating function (e*)’. 


Now we want to determine the exponential generating function of all words 
where all letters appear at least b times. For that, we again use the truncated ex- 
ponential series e,(z). Then we get (e* — e,_1(z))’ as an answer. Observe that this 
is clear if the alphabet has just one letter, and the concept of exponential generating 
functions automatically takes care of the mixing of letters, whence the rth power. 
Variations of this also work. For instance, if we require that all letters appear at most 
b times, then we get e,(z)”, and all kinds of restrictions can be handled. 

Now we consider a balls-in-bins model. Throw at random n distinguishable balls 
into m distinguishable bins. We might think of the balls numbered from | to n. Each 
bin corresponds to one of m letters, and each realization of the experiment is coded 
by a word of length n. Let MIN and MAX represent the size of the least filled and 
most filled bins, respectively. Then 


P(Max <b) =“ [2"lep(<)" =nlletlen(2)" 


and 
P(MIN > b) =n! ([z"] (e/™ @,(2))”. 


Birthday paradox. This is a classical example: Assume that there is a line of per- 
sons entering a large room one by one. Each person is let in and declares her birthday 
upon entering the room. How many people must enter in order to find two that have 
the same birthday? The birthday paradox is the counterintuitive fact that on average 
a birthday collision is likely to take place as early as at time n = 24. Let B be the 
time of the first collision, which is a random variable ranging between 2 and r+ 1 
(where the upper bound is derived from the pigeonhole principle; we assume that 
the year has r days; 2 denotes an alphabet with r letters). A collision has not yet 
occurred at time n, if the sequence of birthdays B,,..., 8, has no repetition. In other 
words, the function B from [1..n] to 2 must be injective; equivalently, B1,...,By is 
an n-arrangement of r objects (=r ordered objects). Thus, we have the fundamental 
relation 

= — ! r 
EASE Sea aes =ntiz4|(1+2)", 


re r 


P(B>n)= 
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The expectation of the random variable B is 


=P Pwe>n=14y ——— 


n>0 


An alternative form of the expectation is now derived, which easily leads to gener- 
alizations. Let f(z) = V, fnz” be an entire function with non-negative coefficients. 


Then 2 s 
Lan! =r f eta = | fOe- 
a Fs 0 0 

(a Laplace transform). Therefore 


(B) = fet (1+*) ar 


Exactly the same reasoning leads to the following: The expected time necessary for 
the first occurrence of the event “b persons have the same birthday” has expectation 


given by the integral 
oo Pree 
Fe(olQya 
0 r 


where the classical case means b = 2. 


Coupon collector. This problem is dual to the birthday paradox. We ask for the 
first time C when f1,..., Bc contains all the elements of 2’; that is, all the possible 
birthdays have been “collected.” In other words, the event {C <n} means the equality 
between sets, {f1,...,B,} = 2. Thus, the probabilities satisfy 


P{C<n}= rit = mle 1)’ =al[z"(e lr _ yr, 


re 
The complementary probabilities are then 
P{C >n} =n![z"| G (= 1)') : 
In the same style as before we get 


tey= f° (1-a-e"y)ar=r @ me. 


j=l 


Alternatively, we might substitute v = 1—e~ /" then expand and integrate termwise; 
this process provides the answer in the form rH,, with harmonic numbers H, = 
1+ 5 feee i. More on these numbers will appear later. This answer can also be 
obtained in an elementary fashion: To get the first copy, you need on average one 
drawing, to get a second one needs r/(r— 1), a third one needs r/(r— 2), and so on. 

The symbolic approach (leading to an integral) has the advantage of straight- 
forward generalizations. For instance, the expected time till each coupon is obtained 


b times is fe 
[ (1-(-ena(s)e"/")")at 
0 
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Permutations and cycles. It is known that a permutation admits a unique decom- 
position into cycles: Let o be a permutation. Start with any element, say 1, and draw 
a directed edge from 1 to o(1), then continue connecting to o7(1), o°(1), and so 
on. A cycle containing | is obtained after at most n steps. If one repeats the con- 
struction, taking at each stage an element not yet connected to earlier ones, the cycle 
decomposition of the permutation o is obtained. This argument shows that the class 
of sets-of-cycles is isomorphic to the class of permutations: 


P = SET(CYC(#)) = SEQ(#). 


This combinatorial isomorphism is reflected by the obvious series identity 


1 ) 1 
l-z/) 1l-z 
The advantage of it is that restrictions are handled in an almost automatic fashion. 


The class A“) of permutations with cycle lengths in A C N and with cycle 
numbers that belongs to B C No has exponential generating function 


P(z) =exp (Iog 


(A,B) (5) — : = zt 2 2 
P“*)(z) = B(a(z)) with a@(z) = y = Bz) = oa 


acA beB 


A popular instance is derangements (fix-point free permutations). The restriction 
is that no cycles of length one are allowed, therefore 


a()=¥% =tx;1-z p@=EF =e 
Qn 4 l—-z : 550 D! , 
leading to 
(we) 
exp ( lo = ; 
P ST e 1—z 


this produces the number of derangements as 


e* z (—1)* 
ni\{z"] =n! yi —_. 
l-z | A 


Notice that the probability that a random permutation of n elements has no fix-points 
is very close (for large n) to , which is a popular result. 


Stirling cycle numbers. The class A”) of permutations that decompose into r 
cycles can be represented as 


P” =SeEtT,(Cyc(#)), 


which leads to 


PW (z) = + (0g —)" 
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(r)_ |n} at Lye 
Pr = ("| =Sei(0e)" 


These numbers are called Stirling cycle numbers; in the older literature, they are 
often called sign-less Stirling numbers of the first kind. This is a somewhat strange 
name, so the new name should be favored as it makes much more sense. 


Therefore we get 


2.4 Partitions and g-series 


Partitions have already appeared briefly before. Here, we want to describe them in 
more detail. 

A partition of a positive integer n is a representation n = i; +i2+---+i, with 
integers 1 < ij <in <---. The i,’s are called parts and k is the number of parts. So 
partitions can be described by the formal expression 


1273", 2 


since n = 1- j} +2- j2+--- with integers j, > 0. If we denote p(n) the number of 
partitions of n and 
P(q)=1+ Y p(n)q" 
n>1 
the (ordinary) generating function of partitions, then we get immediately from the 
formal expression that 
@)=T 


sii 

Recall that the star « is a handy alternative for the construction SEQ introduced ear- 
lier. Note [1] that in the context of partitions it is customary to use the variable q 
instead of z in generating functions. Andrews’ encyclopedic book contains all of 
this, and much more. We also introduce p(0) = 1 to have smoother expressions. Let 
us fix some notation (we assume |g| < 1): 


(q)n = (439)n = (1-4) (1-’)...(1- 9"), 
(q)eo = (43q)0 = (1—q)(1—4")..., 


and, more generally, 


(x)n = (39) = (1—x)(1—2q)...(1—xq"“"), 
(X)oo = (43 g)o0 = (1—x)(1— xq)... 


So we have P(q) = 1/(q).., and also the number of partitions of n into k parts is given 


as 
1 1 
k nk 
q't | - = [q"t 
I 1 tq! Caan 


188 Handbook of Enumerative Combinatorics 


There is a graphical representation of a partition, called a Ferrers diagram. One sim- 
ply codes a part k as a row of k unit squares and arranges them in decreasing order. 
Upon reflecting the diagram at the diagonal, we get another partition, called the con- 
jugate partition. See Figure 2.3 for an example. 


Figure 2.3 
The partition (7,6,4,2,2,1,1,1) and its conjugate (8,5,3,3,2,2,1). 


The number of partitions of n where the parts are of size < k is 


and using the concept of conjugate partitions, this number also equals the number of 
partitions of n where the number of parts are < k. 
The following theorem is very basic for the manipulation of “g-series.” 


Theorem 2.4.1 We have 


(at). (@)n 
F(t)= = t. 
(1) co X (Q)n 
This theorem is attributed to Cauchy in [1] and often called the g-binomial theorem. 


Proof. Splitting off the first factors in the product, we get 


F(t)= —" F(a), or (1—2)F(t) = (1 —at)F(qt). 


Writing F(t) = ¥,s0Ant” and comparing coefficients of t”, we get 


_ n—1 
An —An—-1 = q'An = aq’ 'An-1, or A, = a ee Ay 


1-—q" 


Since Ao = F (0) = 1, iteration of this recursion results in 


as claimed. |_| 
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The first important special case arises from setting a = 0: 


“Vee e 


(too n>0 1—tq n>0 (q)n 


! qe 
(qt)o ASQ (@)n’ 


which is the generating function of partitions of n where the number of parts is la- 
beled by t. Thus, comparing coefficients, we find that 


Dk 


is the generating function of partitions with k parts. The other important special case 
is obtained by replacing a by a/b and t by bt. Then 


(a/b) n(bt)" = (b—a)(b—aq)...(b—aq"!)t". 
Setting now a = —1 and b = O results in 
(tm =P +9") = y Le. 
n>0 n>0 (Q)n 


This is also called Euler’s partition identity. For this we consider the number of 
partitions into distinct parts pg(n). Since each number i can be a part either 0 or 1 
times, the formal equation 


(€+1)(€+2)(€4+3)... 


describes these objects. Thus 


and (—gqf).o is the generating function of partitions of n into distinct parts and k parts. 
Now we replace t by tg in Euler’s partition identity: 


3)+n 


g@ 
(—tq).= [](1+tq") 
n>1 n>0 aa 


Comparing coefficients of t*, we find that the generating function of partitions into 
distinct parts and k parts is given by 
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Euler’s partition identities appear frequently in Analytic Combinatorics and 
Analysis of Algorithms. 

Now we want to compute the generating function of partitions where the number 
of parts is bounded by M and the parts are bounded by N. There is only a finite 
number of possibilities, whence this generating function is actually a polynomial, 
call it G(M,N). Now G(M,N) — G(M — 1,N) is the generating function where the 
number of parts is exactly equal to M; removing one from each part shows that this 
equals g@G(M,N — 1). Together with G(0,N) = G(M,0) = 1, the solution of 


G(M,N)— G(M—1,N) = q"G(M,N—1) 


is given by 
Gin) = Due 
(q)m(@)n 
which is called a Gaussian g-binomial coefficient rae | : 
Once again, we get 
1 
(q)m 


Here is another technique of interest, nicknamed “adding a new slice” by Flajolet. 
Define 


F(q,u) = y y [number of partitions of n with last part iJg”u'. 
n>1i>1 


Now, to create a new slice, i.e., a new part j > i, means to replace u’ by 


Yi (quy = (qu) 


fei 1—qu 
So, taking the partitions with just one part separately into account, 


qu 1 


F a F . 
(q,u) [a aan (q,qu) 
This recursion can be iterated: 
2 3 
qu 1 qu 1 qu 1 
F = stots 
(4.4) tga  1—qull a daa 1—qu  1—q3u 
_y 
1 (qu)x 


Forgetting what the last part is means setting vu = 1; adding | for the empty partition 
results in 


k>0 (Q)k (q)eo’ 
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which follows directly from Euler’s partition identity by setting t = 1. This describes 
the generating function of all partitions in terms of those with exactly k parts, as 
discussed earlier. 

In various applications (see for instance [22]) it is important to expand 


O(x) = (qx)0 = [J  —xq*) 


k>1 
around x = 1, viz. 
Q(x) = 001) +(1)(e—1) + FM G12. 
We have 
; —g* ‘ gitk 
= 00 E ae 2") =200) DE aah 


and therefore 


Or. 1 Q"(1) | 1 2 1 2 
QO(1) raat min = (Lae) Tia 


2.5 Some applications of the adding a slice technique 


A restricted composition of a natural number n in the sense of Carlitz [6] (a Carlitz 
composition) is defined to be a composition 


n=a,+an+---+axg suchthat a; 4aj;+; fori=1,...,k—1. 


We refer to n as the size and to k as the number of parts of the composition. 
Observe that there are 2”~! unrestricted compositions of the integer n with gen- 
erating function z/(1 —2z). 
Let c(n) denote the number of Carlitz compositions of n. In [6], Carlitz found the 


generating function 
C(z) := ¥ e(n)z". 
n>0 
We will rederive this here with the method called “adding a new slice.” This appears 
in [30] and is also described in the book [24]. 

We proceed from a Carlitz composition with k parts to one with k + 1 parts by 
allowing ay; to be any number and then subtracting the forbidden case ay41 = ax. 
In terms of generating functions this reads as follows. Let f;,(z,u) be the generating 
function of those Carlitz compositions with k parts where the coefficient of z”u/ 
refers to size n and last part a, = j. Then 


favs (3M) = file 1) — Files 2u) + [k = 0 fork >0, fo(z,u) =1. 
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The first term means that we forget the labeling of the last part (u:= 1) and add any 
term, together with a labeling by u, and the second one means that we subtract the 
forbidden term, which is a repetition of the previous last part. Introducing F(z, u) := 
Lico1 Se(Z,u) and summing on k > 0, we get 


F(z,u) = F(z, 1) : a ; a — F(z,zu). 
This functional equation can now be iterated: 
F(z,u) = (1+ F(z,1)) 7 (1+F(z,1)) Zu 
1- 1—22u 
ou 
+ (1+ F(z,1)) ig 
now setting uv := 1 and abbreviating 
zi(-1)F! 


we get 
F(z,1) = o(z) + F(z, 1) o(z). 
Since C(z) = 1+ F(z, 1), we find the formula of Carlitz, 


1 
1—o(z) 


The next example is about level number sequences of trees [20]. However, the 
enumeration of these is equivalent to many other objects; the paper [12] describes 
the somewhat erratic history. The objects are sequences (a1, a2,...,a) with positive 
ee a; such that always | < a;,; < 2a;. The number k is arbitrary, and n := a, + 

-- + a,x. The starting value is a; = 1. The interest is in the number H,, of sequences 
satisfying these conditions. Let H,, ‘| be the number of such sequences relative to a 
fixed number k, and let the last cismieht be fixed as well: a, = j. Furthermore, set 


Cz) = 


H\(q,u) > Hig" ui and 
n,j>1 


u) = YA" (gu) 


k>1 


Now we describe how H+!) (q,w) can be obtained from H|(q,u). This is done using 
the substitution 


+ (ug) + (ug)? +--+ (ug) = =“ (1 (ug). 


This means u 
Hl (g,u) = a [H(g,1) —H#(q,u29@?)] 
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and by summing on k (noticing that H'!(q,u) = uq), 


H (qu) = qu+ (Ha!) Hg). 


This can again be iterated: set G(u) = qu+ >“ “GA (4, 1), then 


Ze 
uq ud 
H(q.u) = Gu) — M4 [awa — i [au'a®) 


tT 
q 
46 (cw 8gl4) 


now it is possible to set u = 1, which means that we do not care about the value of 
the last element in the sequence anymore, and get 


7 
H(q.1)=G()- 7 |6@) - 72a |e@*)- a [e")- 


Now this equation can be solved for H(q, 1), with the answer 


(-1)1g?"!-32 


AiG-p0-@)..d-F =) 
(ely ai . 


20 (1—q)(1—@3)...(1-g?-}) 


The last example is about words aja2...da2,41 of odd length where a letter k € N 
is weighted by a geometric probability pg‘! (p+q = 1), i. e., P{aj =k} = pg, 
k > 1, and the letters obey the pattern a, > az < a3 > a4 <---. Let Ton41(u) be the 
generating function such that the coefficient of u! in it is the mass of correct words 


and last letter i. Then we have for n > 1 


H(q,1) = 


peu T. (1) peu 

Tq NY DL £2n—-1 

(1—qu)(1—q?u) (1—qu)(1—q?u 
pu 

l—qu 


Ton41(u) = Ton1(q°u), 


T\(u) = 


Adding a new slice means adding a pair (k, j) with 1 <k <i, j >k, replacing u! by 
1 and providing the factor u/. But 


i 2 2 : 
pa dY pqi lu — —— 5 (eu) 


Sh ~ (l-qu(i—@u) (i qu)(1— eu 


which explains the recursion. The starting value is just 


ly = 
La 
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We introduce the generating functions 


F(z,u) = y Tonii(uye"t! and f(z) =F(z,1). 
n>0 
Summing up we find 


puz peu 


sc eee aay aro) 


F(z,1) 


1—qu 


Iterating that we find 


ib pee ree 
pq? 


and eventually 


= (—1)"(pz)*"*! n(n+1) 
f(z) —_ y q . y q)2n 


(= Bg (pz)°" n(n—1) 

qd ‘ 
n>0 (q)2n+1 n>0 ( 
This example was taken from [44]; in this paper it is also explained why the limit 
for g > 1 of f(z) is the tangent function tanz. It is a classical result that tanz is the 
exponential generating function of up-down (or down-up) alternating permutations 
of odd length. 


2.6 Lagrange inversion formula 


Let y = x®(y), where ®(y) is a power series such that B(0) 4 0. It is obvious to 
expand x as a power Series in y, but we want just the opposite. This is the celebrated 
inversion formula of Lagrange. We give three versions of it. 


1 n 
"ly = —b*'(®0)) 5 Grey): 
Slightly more general (for n > 1, p > 0): 
"ly? = Ep"? (@0))". 


Even more general (for n > 1 and a power series g(y)): 


i"le) = -p" He) (@0))”. 


n 
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Proof. | We use Cauchy’s integral formula. In case this is not justified analytically, 
it can be done on a purely formal level as explained in [25]. The present approach is 
also easy to remember. Observe that 


@(y) — yO! 
ay oe (y) 


Now 


1 dx 
[x”]y? = afm 
el D(y) — yb'(y) "FH (y) 
- sai? Oey) yt 
1 1.y\ ® | (y) 
= =a f o(20) —y® (”)) THis 
= [y"-P (£0) -y0'0) o""0) 


= fy" ?]&"(y) — p" Je’) @"""'09) 
= fy P]@"(Q) — <p] (@"))' 


= fy P]o"(y) — prj") 


Now we set 


(vy) = ¥ epy? 


pel 


and get 


"Is0) = L cpl"? = Y ep 2b" 0)" 


p2i p2l 
= bP NE pe? '(@0))" = 1b" 'e'0)(@0)" 
pa 


Applications. (The following tree structures have been introduced in Section 2.2.) 


t-ary trees. These objects are recursively built from a root and ¢ successors, which 
are themselves t-ary trees. A tree might be empty as well. For t = 2, we get the 
important special case of binary trees. The equation B = 1 +.B" for the generating 
function, counting trees according to the number of vertices, is immediate. Set B = 
1+, then y = x(1+)’, in order to make the Lagrange inversion formula applicable. 
Then ®(y) = (1+,)’, and thus 
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Number of leaves in planar trees. The recursive description of planar trees im- 
mediately translates into a bivariate generating function: 


zG(z,u) 


G(z,u) =zZU+ T—G(z,u)’ 


the variable u counts leaves, z nodes (in planar trees). Note that for uv = 1 and y = 


G(z, 1), we have y = z®(y) with ®(y) = 74). 


1 n 
Gna = [w'][21G Gu) = be] pt] (u+ >) 
1 _ n n—k 
=<p4(")_ _ 
e k} (1—y) 
_1fn\) ea 1 
er @) by Gyre 
_ 1 /n\ (n—-2 
“a ee dye 
These numbers are sometimes called Narayana numbers. 


Planar trees according to degree of the root. Let P,;, be the number of planar 
trees with n nodes and root degree k. Then, again, y = z®(y) with ®(y) = y/(1—y) 
enumerates planar trees. Let us assume that p > 1 and n > 2, since P,,o = [n = 1]. We 


find 
k 2n—3—k 
n—1—k —n+l1 
pies j= = 
Ly jay) ( Had ) 


n—-1 


2.7 Lattice path enumeration: the continued fraction 
theorem 


We follow here [24] and [13]. 


Definition 2.7.1 (Lattice path) A Motzkin path v = (Up,U\,...,Un) is a sequence 
of points in the discrete quarter-plane No x No, such that U; = (j,y;) and the jump 
condition |yj41 —yj| < 1 is satisfied. An edge (U;,Uj+1) is called an ascent if yj+1 — 
yj = 1 a descent if yj+1 —yj = —1, and a level step if yj; —y; = 9. A path that 
has no level steps is called a Dyck path. The quantity n is the length of the path, 
ini(D) := yo is the initial altitude, fin(V) := yy is the final altitude. A path is called 
an excursion if both its initial and final altitudes are zero. The extremal quantities 
sup D := max ;y; and inf v := min; y; are called the height and depth of the path. 
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A path can always be encoded by a word with a, b, c representing ascents, de- 
scents, and level steps, respectively. What we call the standard encoding is such a 
word in which each step a, b, c is (redundantly) subscripted by the value of the y- 
coordinate of its initial point. For instance, 


CO do 41 a2 bz C2 C2 a2 by by by A C1 


encodes a path that connects the initial point (0,0) to the point (13,1). 


Let us examine the description of the class written HSN of Motzkin excursions 
of height < 1. We have 
<l] ny * <l 1 
Hy = (co!” => Hoy’ =7—- 
0 


The class of excursions of height < 2 is obtained from here by a substitution 
cot co +.a0(c1) "bi, 
whence 


[<2] i 
dob, 


l—co-— aa 


Iteration of this simple mechanism yields the finite version of the continued fraction 
theorem of Flajolet [13]. 


Theorem 2.7.2 (Continued fraction theorem, finite version) 


1—cp_y 
The unrestricted version leads to an infinite continued fraction. 
Theorem 2.7.3 (Continued fraction theorem, infinite version) 


1 
Hoo = 


agb, 


1 
se ab 
l-c- 
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Generating functions written in this way are nothing but a concise description of 
usual counting generating functions: for instance if individual weights a, Bj, yj are 
assigned to the letters a;, b;, c;, respectively, then the ordinary generating function of 
multiplicatively weighted paths with z marking length is obtained by setting a; = az, 
bj = Biz, cj = Ye. 

The “numerator” and “denominator” polynomials, denoted by P, and Qj, are 
defined as solutions to the second-order (or “three-term’’) linear recurrence equation 


Yna1 = (1—cn)¥n—an—1baYn-1, >= 0, 


together with the initial conditions (P_1,Q—1) = (—1,0), (Po, Qo) = (0, 1), and with 
the convention a_; = bo = |. These recursions are easy to obtain by replacing | — 


Cn—1 by 1 —cy-1 — sar ibn and comparing numerators and denominators separately. 
These polynomials are also known as continuant polynomials [26, 48]. For the 


computation of HM and P;, Q,, one classically introduces the linear fractional 


transformations i 


20) — 
J 1 —c;—ajbj+iy 
so that 

Hho = gyog10---0g4-1(0). 


Linear fractional transformations are representable by 2 x 2 matrices 


ay+b a b 
t> 
cy+d c d 
in such a way that the composition corresponds to the matrix product. By induction 


on the compositions that build up HEM, there follows the equality 


Ph, — Ph—1an—1bny 
Qn — Qn—14n—1bny 


Eventually, one sets y := 0. The polynomials P, and Q;, both satisfy the recursion for 
Y;, as just given. 

Now we come to applications. In order to count Dyck paths, it is sufficient to 
substitute 


80°81 0°*O8p— 1) = 


ajhZ, bjHez, cj 0. 


Because of the natural bijection (sometimes called the glove bijection), described 
earlier, a Dyck path of length 2n and height / translates into a planar tree with n+ 1 
nodes and height h-+ 1, so that the results translate directly; compare [10]. In order 
to avoid misunderstandings, we state explicitly that the height of a planar tree is the 
length of the longest path from the root to a leaf; the length of a path is counted 
in terms of the number of nodes on it. This is the original definition used in [10]; 
sometimes people count the number of edges, which is then one less than what is 
considered here. The height of a Dyck path is of course the maximal vertical level 
reached. 
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The families of polynomials P;,, Q, are in this case determined by a recurrence 
with constant coefficients. Define the Fibonacci polynomials by the recurrence 


Fn42(z) = Froi(z) —2Fa(z), Fo(z) =0, Fi(z) =1, 


then it turns out that Q,(z) = F,41(z7) and P;,(z) = F;,(z*). The Fibonacci polynomi- 
als admit an explicit form 


ee 


If we take the limit 1 — ce in 


Fi,(z) = 


Fi, (z) 
Frii(z)’ 


then we get 


D(z) = 


1-V1—4z 1 2 “ 
ee Se eR ie y z”, 
2z ott l\a 
the generating function of Dyck paths of halflength n; the coefficients are Catalan 
numbers. The limit is taken in the sense of the “discrete topology,” but it works as 


well as the “pointwise” limit for analytic functions. As demonstrated in [10], all our 
expressions become easier with the substitution 


SiS gee 
2 +a? same (Se ae 
Then, by solving the second order recursion and substituting, 
1 1—u" F, 1—u' 
——_., —— = (l+u)——_. 
1—u(1+u)'! Fri 1—y*t! 


F, = 


The limit of the last expression for h + © is 1+ u. The function D(z) — 
_ 1—u2 A+ 


Fi,(z)/Fh+i(z) = 4" 7"Saar is easier and describes the Dyck paths with height > h, 


according to halflength; call it H|=/I(z). 
The following method to extract coefficients is in [10]; it is the Lagrange inver- 
sion formula in disguise and uses the Cauchy integral formula: 


oe ad h 
[tH eH (2) = 1 f dz 1 u u 
Zz 


omit ew 1—uh 
=e gue tu)""(1-n) 1-2 ub 
ni yrtl u  1—uh 
= uw" "1 —w)?(1+u)" ¥ ul (2.1) 
k>1 
= Paper —2u +u’)(1 ag)? 
k>1 


es (ee) =2{ at) i ee 
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We would like to introduce an alternative method to compute the generating func- 
tion of Dyck paths of height < h; it appears for instance in [41]. Define @;(z) the gen- 
erating function (according to length) of non-negative lattice paths starting at (0,0), 
ending at (n,i), and height < h, fori =0,...,4—1. Then 


Po(z) =1+201(Z), Pn—1(Z) = ZPn-2(2), 
0;(z) = z@;-1(z) + z2Qi41(z) forl <i<h—2. 


This system is best written as a matrix equation: 


Po 1 -z O.... o 1 
Q) —Z 1 FIZ as — 0) 
i= Waele 
On-1 -—z 1 Oh—-1 0 


It can be solved using Cramer’s rule. Denote by a; the determinant of the matrix 


with h rows and columns. Expanding with respect to the first row yields the recursion 
An = an =F oi -5; forh>2,a9=a,; = 1. 


The solution is 


an = 


1 (ere = 


V1—422 2 2 
and thus ; 
Zah—\—i 
Og) 
ap 
The good substitution in this case is z= Ber because then 


1 _ yenr2 


Oh Te ee 


This approach works also in the Motzkin case (level steps allowed), see [42]; the 
equation is then 


Ph—1 —zZ 1l-z} \@r-1 0 
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the recursion for the determinants is 


Gy = (1 —z)an_1 —2°ap-2, 


and 


an 


1 (aa) | 
VI- 22-32 2 2 , 


and the good substitution is z = Explicit expressions are also possible; they 


Vv 
I+v+v2" 
involve trinomial coefficients ee ) = [*](1 ++)" (notation from [9]). 

It is worthwhile to notice that Motzkin paths can be obtained from Dyck paths 
by squeezing in arbitrary sequences of level steps between consecutive steps. That 
amounts to replacing z by z(1+z+---) = T-. if we assume that an up/down-step is 
followed by a sequence of level steps. We must provide for an arbitrary sequence of 


level steps in the beginning. So we get 


1 (( z ))\)-be ee 


l—z l-z 222 : 


as expected. Since this procedure does not affect the height, functions like H(" also 


translate. 


2.8 Lattice path enumeration: the kernel method 


This is taken from the survey paper [45]; I am confident that the present Handbook 
will have much more about the subject in different chapters. 

In the present author’s view, the kernel method originated in Knuth’s book [31], 
where it was presented as an innocent exercise (Exercise 2.2.1.4). Later, it was turned 
into a method; see [4] and the literature cited therein. It was probably rediscovered 
independently by many people; I recommend to follow the references in [4]. 

I feel that I cannot do anything better as an introduction than to reproduce Knuth’s 
original exercise. One starts at the origin, and can advance from (n,i) to both (n+ 
1,i+1), except in the case when i = 0, when one can only go to (n+ 1,1). In this 
way, one models non-negative lattice paths (or random walks). The Dyck paths of 
the previous section are the case where one ends at level 0 after n steps. We want to 
know how many paths lead from the origin to (n,0), and, more generally, to (n,i). 
(Clearly, this is a very classical subject, but the derivation that Knuth presented is the 
subject of this presentation.) One uses generating functions f;(z), describing walks 
leading to (n,i); the coefficient of z” is the number of walks from the origin to (n, i). 
The following recursions are immediate: 


fi(z) 
fo(z) 


zfi-1(z) + 2fit1(Z), i=l, 
1+zfi(z). 
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Now one introduces F (z,x) = Lnso fn(z)x”, multiplies the recursion by x’, and sums: 


F(z,x) — fol) = aF (x) + =[F(e,4) — fol) —fi2)], 


or 
F(z,x) = 2xF (z,x)+ z [F (z,x) — F(z,0)] +1, 
whence “F(z,0)—x 
Ae ae 


Plugging in x = 0 leads to nothing, but the denominator factors as z(x — r1(z))(x — 


ro(z)), with 
izV1—42 
2z 


Note that x— 1) (z) ~x—zasx,z— 0. Therefore the factor 1 /(x— 1) (z)) has no power 
series expansion around (0,0), but F(z,x) has, so this “bad” factor must actually 
disappear, i.e., (x—71(z)) must be a factor of the numerator as well, which leads 
to the equation zF (z,0) =r) (z), from which F(z,0) can be computed. Consequently, 
F(z,x) is then also explicitly computed, and the factor (x — r)(z)) can be cancelled 
from both numerator and denominator. 

From this, one finds for instance that [z°”]F (z,0) = nal ( 
lan number, and similar expressions for [z"x']F (z,x), for n = i mod 2. 

The next example revisits the toilet paper problem, a popular subject introduced 
by Knuth [34]. He considers two rolls of tissues, with m and n units, respectively, 
and random users, who are with probability p big-choosers (taking one unit from 
the larger roll) and with probability g = 1 — p little-choosers (taking one unit from 
the smaller roll), respectively. The parameter of interest is the (average) number of 
units remaining on the larger roll, when the smaller one became empty. 

Let m be the number of units on the larger, and n on the smaller roll; Myn is 
the expected number of units left on the larger roll, when the smaller one becomes 
empty. 

The recursions are 


r12(z) = 


” a well-known Cata- 


Mmno =m, 
Minn = Minn-1; m> 1, 
Min = PMn-1n + qMinn-1, m>n>0. 


Define 


Fo(z) = 3 Minne F, (z) — y? Minm—12""- 
m>0 m>1 
Note that 
Fo(z) = y Minmz = y Minm—12" = Fy (z). 


m>0 m>1 
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Define 
F (z,x) = y Mas. 


m>n>0 


Then, by summing up, 


F(z,x) = y Minpaz x"? + y Mmoz"x" + y MinnZ” 


m>n>0O m>0 m>0 


= LX 
y [PMin—1,n +qMmnn—1\e"x" mo =e + Fo(z) 


m>n>0 (1 oo ) 
= PxX 3 Mia 


m—1>n>0 


+4 y Mig ac OO) & 


I 


a 4+ Fo(z) 
m>n>0 (1—z) 


= pz [Fex) - >: Mmoz"x") 


m>0 
= xX 
+ = a MinnZ"X" mae 71 ov\2 + Fo(z) 
Xm>n>0 (1 ~ 2x) 
xXx 
= pzx|F(z,x) — — 
palF(a3)- qo 
+ 4 [Fix) —Xx y Martine’! = y My n2"] 
x n>0 n>0 
xXx 
+ —— +k 
(1—zx)? Oe) 


= perk (2.x) + 4 F,2) —xAi(2) Fi @)| 


+ ESP + il) 


(Here, we used the ad hoc notation a > b:< a—b > 2.) Solving, 


(1=zx)? (1—zx)2 


att + @[l—q—a/x]_ Fi@la— px] — 2 


F(z,x) = = 

oe 1 — pzex—q/x pur —xt+q 
2(4— 

Fi(2)[q— px] — 


~ p2(x—ri(z))(*— r2(z)) 


b 


with 


1+ V1 -4pqz 


ria(z) = Ope 


Therefore, for x = r}(z), the numerator must vanish, yielding 


zrt(z)(1 — pzri(z)) 
(1 —zry(z))? 


Fi(z)[q— pri(z)] =0, 
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7 __ai@(-pei) _ 
M2) Gor Qari (Tape FP) 
with ps) 
C(z) = eae 
Note that rj(z) = Cl) and that 1/r2(z) = ri(z)pz/q. The argument that 1/(x — 


pz 
r1(z)) has no power series expansion around (x,z) ~ (0,0) must be replaced here by 


something else, for instance, that F (z,x)/z ~ 1 for (x,z) ~ (0,0), which follows from 
the combinatorial description of the problem. 

The expression for F (z,x) is ugly, but we can extend Knuth’s asymptotic analysis 
to Minym—n for m — 9 and fixed n; the instance n = 0 was given in [34]. This asymp- 
totic analysis is perhaps best understood after consulting Section 2.17 first. However, 
we decided to put it in here, so that we avoid having to repeat the description of the 
problem later. 

For g < p, Knuth has shown that the local expansion of C(pgz) around z = | 
starts like 
3(z—1)?+--- 


Hence the local expansion of F (z,x) around z = 1 is given by 


1 P 
: + Q(z), 
1—z (2p—1)(1—x) QX:) 
where Q(z) has a radius of convergence > 1. So Mnm—n = p/(2p — 1) + O(r”) (for 
a suitable 0 < r < 1), and the n plays no role here. This is intuitive, the big-choosers 
dominate, so it does not really make a difference whether the second roll is slightly 
smaller. Now let us assume that p < q. Then F(z,x) starts like 


1 p= 
(=z) (p=1)0.—») 


1 1 P p(l—p) 
Al T—x* q(l—xp or nem | 


The coefficient of z’” is asymptotic to 


12> 1 Pp p(l—p) 
ioatoa | aoe oe 


And the coefficient of x” (n fixed) in this is 


(m+1)- 


1—2p | Pp P (P ‘| 
nib): eS at ( ) 
(m+1) ae a ) pa 


or 


poe ty Oe 2 
q q 1—2p\q 
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For n = 0, we find again Knuth’s value tem + ra, Perhaps it is not very intuitive 
at first glance why this grows with n. However, for larger n, the process tends to be 
over more quickly, and so more will be left on the large roll. 

Now let us discuss the case p = q. Then 


C(pqz) = a Le; 


1 1 
2 2 


and * i 
PO 6 EE ag) (Re 
F(x) ~ (1-2) 9? (1-2) 


and the coefficient of z behaves like 


Oa ayaa) Ia yew Oo 


Furthermore the coefficient of x” (n fixed) in this is 


m 3 1 (n+1)(n—2) 
nfs 7+ 2 ee) 


For n = 0 we find again 
m 1 
2 ?= 3 
/ ma 4,/am 


Again, since everybody takes at random, the process tends to be over more quickly, 
leaving more on the larger roll. 


2.9 Gamma and zeta function 


These two special functions appear in many contexts; therefore we only collect here 
a few basic facts. General references are [49] and [24]. 
Euler defined 


T(z) =a e't'dt for Rz>0. 
0 


It is easy to show via integration by parts that '(z+ 1) =zI'(z) andI'(1) = 1, whence 
I'(n+1) =n! for positive integers n. There is another definition due to Gauss, 


rc i nink 
is novem 2(Z- 1)...(z-+n) 


*We use the symbol ~ in the sense that a full asymptotic series in powers | — z would be available, at least 
in principle, to which the method of singularity analysis of generating functions (transfer theorems) 
[19] is applicable. See also Section 2.17. 
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zlogn 


The meaning of n“ is here e , with logn being real and positive. Now 


act 1)... +n) = e-Hene(1 42) (142) (1 3) 


niezlogn 1 2 n 


a etsloane TT (1 xis = est. 
k=1 k 


Here, H, = 1+ 5 tere tf i is a harmonic number. Thus, the limit can be made 
explicit by writing 
1 Zz 
= y ( ad “) | 
ze e ' 
T(z) I] n 


n>1 


with Euler’s constant 


y= lim (Hy —logn) = [- —log (1+ ~)| = 0.5772156649, 


n>1 


Gauss’ definition is more general, but coincides with Euler’s when both make sense. 
From it, it is not hard to derive Legendre’s duplication formula 


T(z)P'(z+ ) = (22) /n2!-* 


as the reflection formula 


rer 5 


From this, the special value (5) = /7 follows. 
The logarithmic derivate of the Gamma function is 


ve) i= 


using the product form it can be written as 


yet) =-9-¥ (4) Fee, 


nal n+zZ Nn 


with the Riemann zeta function ¢(s) = ¥,5,n-* for Rs > 1. This also expresses 

harmonic numbers: H, = w(n+1)+/. From the last expansion of y(z+1), the 

expansion of I(z) around any integer can be reconstructed; for example (z+ 1) ~ 

1— yz for z + 0; the Gamma function has simple poles at z = —k, with residue 

(—1)*/k!, k =0,1,2,.... This can be seen directly from Gauss’ definition as well. 
For purely imaginary values, we have 


qT 


Ne: zsinh(zz) ’ 


which also shows the rapid decay when z becomes large. 
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The Riemann zeta function is for Xs > 1 defined by 


C= De 


It can be continued to the whole of C and has only one simple pole at s = 1 with the 


Laurent expansion 
1 
C(s) = rs 


The following special values are useful: 


€ (2k) = (24)! fork EN, 
g(-2e+1)=-F#, 6(-28) =0, 
G(s) ~—5 —logV2a-st-, (s > 0), 


with B,, the Bernoulli numbers. 
The following functional equation due to Riemann is very famous: 
s 1-s 


r() C(s) = nT (—*) C(1—s). 


Some properties of the Bernoulli numbers (taken from [26]) are now collected: 
They are defined by their exponential generating function 


from this the recursion 
a 1 
y B= In=0] foralln>0 
imo \ & 


follows. Bernoulli numbers with odd indices, except By = — 5 are zero. The coeffi- 
cients of the tangent may be expressed by them: 


tanz= Y(—1)""'4"(4" — 1) By ——. 
Xu "(2n)! 


And the celebrated sum of the first mth powers is 


1” $2" ot (= 1)"= (Bm+1(1) —Bm41). 


m+1 


Here we have Bernoulli polynomials defined by 


Bn(x) =¥ ("’) Byx"*, 


k 
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They satisfy B,,(0) = B,, and have the exponential generating function 
XZ 


ze 
YL Bm(x) ml = @oT" 


m>0 


2.10 Harmonic numbers and their generating functions 
We start with 
1 


1 n k 
—— log —— = 
a LE 


1 “1 
i log —— = — =A). 
i" jloeg = hah 


For many applications it is necessary to know the coefficients of 


ft avg d 
(l—zjmt! 08 L= 


for integers m,k > 0. The following derivation is based on [50]. We start from a 
bivariate generating function 


: log* ra = 1 (1 1 ) 
- 2 ex og —— 
Bo (1 zymrl e l1—zk! (1—z)"t P & ie 


1 m+t-+n 
= (1—zymt FP a a ‘ je 


n>0 


("eT +c) 
-E(" a (L814 555)) 

"e eo (ES - Lor) 
- ECT) ee (Eos 


i>l 


Now we can read off coefficients: 
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1 ¢k m+n 
nk k == k 
k3[z" qa pari log aA =( i Jan 
: (nitty — HO)? (Hy — HOY 
ieee nek Li Qh... jyljgl--- 


For k = | and k = 2 we get the important special cases 
1 1 
] — 
(l—z)mt! os ey 


‘“ : ") (Hoshi i Ey) ) 
n>0 n 


1 1 + 
(arr oe = E(", ") [eins Hy)’ — (Hate —Ha) |. 


The general expression can be written using Bell polynomials [50]. 


2.11 Approximation of binomial coefficients 


One often needs to approximate binomial coefficients (ae) in a central region (e.g. 


|k| < ,/nlogn). We obtain, for instance by Stirling’s formula (2.4), 


(7”) 
A derivation of Stirling’s formula will be given later (2.4). Quite often rth differences 
of binomial coefficients appear, like (for k = 2) 


(121) = 2024) + (2) 
) 


Let us recall the difference operator A (operating on k): 


= Ge 7 (;) oe @ a): 


Let us consider A”e~*’/”. It is not difficult to see that one can apply this difference 
operator term by term to the expansion (2.2). 
For this, we need the Hermite polynomials. Here are the necessary properties [2]: 


(re) we Win. ( ke = is J (2.2) 


2-1? t” 
e* = y H(z), 


n>0 
Ho = 1, A, = 2z, An+1 = 22H, — 2nHy-1, 
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Thus 


eee (jew 
me F iy ay (‘) Lm alee 


We know (with Stirling subset numbers) that 


Ley (;) ym afm, 
Hence 


(<1) are Blt ws eB l#-(+) 1 oe = 1 Fe]. 


Therefore we have 


: 2n 
ven (7) nate aS «Phn,( =) = 


which was announced in [21]. 
To obtain more terms we have to consider A’k’e 
general formula (of Leibniz type) 


2 F 
“ke /n For this, we can use the 


a’ (reat) = ¥ (7) (a re+0) (ale) 


with f(k) =K and g(k) =e", 

A very general answer that includes the approximation of binomial coefficients 
as a special case is presented in [27], which we sketch here; compare the original text 
for some technical conditions. 

Let 

g(z) = DY pet 


k>0 
be a probability generating function, and its Thiele expansion given as 
071? —k3t?—kyt# 


EN x= | fe aia a 
a(e!) = exp (ur + op 8 ap cay ), 
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where pt = g/(1) and o? = g""(1) +9'(1) —(g’(1))* are expectation and variance. The 
constants &, are called cumulants. Further, let 


Ank = [zt] 9(z)", 


where k is chosen to make the exponent an integer, which is “not too far away” from 
the expected value tin. 


For our application, we choose g(z) = 1k, so that W = 5 and 0? = ie Then 


(nie) 


Q2n ’ 


Aonk = 


it does not matter whether we normalize by 27” or (yg since, by Stirling’s for- 


mula (2.4), . ‘ ; 
n 
q-2n _ —5/2 
a) Jan  8/nn3/2 One 


where any number of terms would be available. 
The answer in [27] is 


Z k3 k k3 k3 


1 k 
Anko (som) |! ie 
ad ov2nn oEP 202n 204 n as 606 n2 Ole ) 


there is also a general version given, including terms of the form kX /n”, which we 
do not reproduce here. 


2.12 Mellin transform and asymptotics of harmonic 


sums 
This section is based on the survey [16]. For details and proofs we refer to this clas- 
sical paper. 
The Mellin transform associates to a function f(x) defined over the positive reals 
the complex function f*(s) where 


P(s)= [ fe) lax 


it is a close relative to the Laplace and Fourier transform. 
The major use of the Mellin transform examined here is for the asymptotic anal- 
ysis of sums obeying the general pattern 


F(x) =) Ac (Mex), 
k 


either as x —> 0 or as x —> oo. Sums of this type are called harmonic sums; f(x) is 
called the base function. 
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Harmonic sums surface at many places in combinatorial mathematics as well as 
in the analysis of algorithms and data structures. De Bruijn and Knuth are responsible 
in an essential way for introducing the Mellin transform in this range of problems, as 
attested by Knuth’s account in [32] and the classic paper [10], which have been the 
basis of many later combinatorial applications. 

By a simple change of variable, we find 


Fs) = Daan S°0) 


It is this factorization property that makes the Mellin transform useful for harmonic 
sums. The Mellin transform exists in a fundamental strip (—u,—v) := {z€ C | —u< 
Rz < —v}, if 


f(x) =O(x") asx 0, f(x) =O(x") asx, 


As the integral defining f*(s) depends analytically on the complex parameter s, a 
Mellin transform is in addition analytic in its fundamental strip. 

For instance, the function f(x) = (1 +.x)~! is O(x°) at 0 and O(x~!) at infinity, 
hence a guaranteed existence strip for f*(s) is (0,1), which here coincides with the 
fundamental strip. In this case, the Mellin transform may be found from the classical 
Beta integral to be f*(s) = =5.., which is analytic in (0,1). 

The function f(x) =e satisfies 


e*~1 asx—0, e*=O(x°) for any bas x. 


Therefore the Mellin transform 
f(s) = [ e*x°-ldx =T(s) 
0 


exists in (0,0) and is analytic there. 

Let H(x) be the step function defined by H(x) = [0 <x < 1]. Then H*(s) = t in 
(0°). 

The following list covers the essential functional properties. 


f(x f(s a, 
x" f(x) fi(s+v) (a—v,B—v) 
f(x") at Gp) (ap,Bp) (p >0) 
(4) =fit3) (—B,—a@) 
f (ux) bh *f*(s) (a, B), (u > 0) 
Le Ac (Mex) | Le Ace? f(s) 
f(x) logx £ f*(s) (a, B) 
41%) =S7(5) 
aie) | = wales 1) 
Jo f(t)at —1f*(s+1) 
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The empty ranges depend on the situation. They are usually the intersection of 
the strips of the ingredients, like the Dirichlet-like series and the strip for the base 


function. For instance, let 


then 
F*(s)=(1+2%+37%+-+-)F(s) = ¢(s)I(s). 


It is valid in the intersection of (1,°°) (¢-function) and (0,°°) (I-function), so it is 


(1,°). 
Here is a little list of some common Mellin transforms. 


e* Is) (0,20) 
e*—] T(s) (—1,0) 
e*—l+x T(s) | (—2,-1) 
2 
e Ir) | (O.=) 
T+x aaah (0, 1) 
log( +x) can (—1,0) 
H(x)=l0<x<1]] “TP” | (0,2 
a k (=1)*k! 
x (log x)"H (x) (sta)yAl (—@, °°) 


Theorem 2.12.1 (Mellin’s inversion theorem) 
(i) Let f (x) be integrable with fundamental strip (a, B). If c is such that a<c< B 


and f*(c + it) is integrable, then the equality 
1 C+ic° 
|. Foxtas= Fl) 
c 


2ni 
holds almost everywhere. Moreover, if f (x) is continuous, then equality holds 


everywhere on (0,°). 
(ii) Let f (x) be locally integrable with fundamental strip (a, B) and be of bounded 
variation in a neighborhood of xo. Then, for any c in the interval (a, B), 
ee pe £0) tf) 
li wats ok 4 —s c= 0 0 Y 
Toco 201 Pees 2 


Recall that a function of bounded variation is a real-valued function whose 
total variation is bounded; the total variation of a real-valued function f(x), 


defined on an interval |a,b], is the quantity 


sup) f(ai+1) — fa)|, 


P 0<i<np 


taken over all partitions P = (xo0,...,Xnp) of the interval [a,b]. 
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We need the notion of a singular expansion. Let @(s) be meromorphic in Q, and 
let Y include all the poles of @(s) in Q. A singular expansion of @(s) in Q is a formal 
sum of singular elements of @(s) at all points of .7. When E is a singular expansion 
of @(s) in Q, we write @(s) < E, s € Q. This formal expansion is a concise way of 
combining information contained in the Laurent expansions of the function (s) at 
various points. For example, 


is the singular expansion in the whole of C. 


Theorem 2.12.2 (Direct mapping) Let f(x) have a transform f*(s) with nonempty 
fundamental strip (a, B). 
(i) Assume that f(x) admits as x — 0° a finite asymptotic expansion of the form 


f(z = 3 cz yx? (logx)« + O(x”), (2.3) 
(,k)eA 


where the € satisfy —y < —& < a and the k are non-negative. Then f*(s) is con- 
tinuable to a meromorphic function in the strip (—Y,B) where it admits the singular 
expansion 


_ F (—1)*k! v sti. 
fax de, Ek(gp Eye tC iF E ( 7,B). 


(ii) Similarly, assume that f (x) admits as x — © a finite asymptotic expansion of 
the form (2.3) where now B < —§ < —y. Then f*(s) is continuable to a meromorphic 
function in the strip (a@,—¥Y) where 


fox- ¥ eagepgyet + Ol" sé (a,-7). 


Thus terms in the asymptotic expansion of f(x) at 0 induce poles of f*(s) ina 
strip to the left of the fundamental strip; terms in the expansion at © induce poles in 
a strip to the right. 

This principle is general: Subtracting from a function a truncated form of its 
asymptotic expansion at either 0 or oo does not alter its Mellin transform and only 
shifts the fundamental strip. An instance is provided by the functions e~*, e~* — 1, 
e-*—1+- x, all having the Mellin transform I(s), in different strips. 

Under a set of mild conditions, a converse to the Direct Mapping theorem also 
holds: The singularities of a Mellin transform that is small enough toward ico encode 
the asymptotic properties of the original function. See [16] for a precise statement. 

The Mellin summation formula is 


LAS (ua) ~~ y Res(f* (s)A(s)x*), 


scH 


with A(s) =Yy Age; 
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For an expansion near 0, the sum is over the set H of poles to the left of the 
fundamental strip, and the sign is +. 

For an expansion near oo, the sum is over the set H of poles to the right of the 
fundamental strip, and the sign is —. 


Example 2.12.3 (Harmonic Numbers) We refer here to Section 2.9 about proper- 
ties of the zeta function and the Bernoulli numbers. We write 


pd GS eee ae 
me) = EG oe) aa 


and notice that H, = h(n) and that h(x) is a harmonic sum, with Aq = Ux = i . We 


have 
10 


h'(s)=-—"_£(8), 


sin ws 


and the fundamental strip is (—1,0). The singular expansion to the right of this 
fundamental strip is 


Hence 
(—1)*B; 1 
k nk 


1 
H, ~lognt+y+ oe dX 
The dominant terms come from the expansion at 0. We have to take the residue (with 


a negative sign) of 
Xu 


sin 7s 


C(s)x*, 


which is 
[st] (-s? + (logx+y)s~!+-- -) =logx+y¥. 


The full expansion for the harmonic numbers is 


Box 
2kn2k ° 


1 
H, ~logn+y+—-Y) 
2n il 


Example 2.12.4 (Stirling’s formula for the [-function) From the product decom- 
position of the Gamma function, one has 


I(x) =Tet)-w=P [= toe (1+)]. 


One computes 
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with fundamental strip (—2,—1). There are double poles at s = —1, s =0. and simple 
poles at the positive integers. The main contribution is 


Tu 
ssin(zs) 


[(s+1)"}E(—s) x = xlogx+x(y- 1). 


The full expansion is 


Bon 1 
(2n — 1) x2n-1° 


log(x!) ~ log (xe*V2nr) oie (2.4) 


n>1 


Example 2.12.5 (A divisor sum) Consider 
D(x) = y d(kje™, 


k>1 


where d(k) is the number of (positive) divisors of k. Since 


we find 
D*(s) =T(s)¢7(s), 


and the main contribution of the asymptotic expansion at x = 0 is given by 


_ —logx+y7 
= : 


[(s—1) Ps) g7(s)x-* 


Example 2.12.6 (Height of planar trees) We have seen (2.1) that the probability 
that a Dyck path of length 2n has height > h—1 is given by 


y Cia) = (niu) sr (ape) ; 
ros mile) 


Because of the fundamental correspondence between Dyck paths and planar trees 
this enumerates as well the family of planar trees with n+ 1 nodes and height > h. 
In order to compute the expectation, one has to sum this on h > 1, which leads to 


2n 2n on 
Envi = (n+1) y d(k) (nsx) At BE tee) 
kel s 


Here, again, d(k) is the number of (positive) divisors of the integer k. Approximation 
of the second difference of binomial coefficients as described earlier (Section 2.11) 
leads to the approximation 


2 
Eni ~nYy die *l" (= = 2) 


k>1 
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The asymptotic evaluation of this series is a typical application of the Mellin trans- 
form. With x = 1/,/n, we need to estimate 


¥ d(ke** (42x? — 2), 
k>1 


which is a harmonic sum, and its Mellin transform is 


Y d(kyk*: Lo (4x? 2) de = 67(s)(s- (5). 


k>1 


Using the inversion formula, this sum can be written as 


; / ec ce Ir (5) tas, 


271 J2—j00 


The residue at s = | (after shifting the line of integration to the left) produces ve 
which is ,\/7n, which is the asymptotic equivalent of the average height of a planar 
trees with n nodes. For more details, compare [24] and the literature cited therein. 


Example 2.12.7 (A doubly exponential sum and periodicities) The prototype of 
harmonic sums with a fluctuating behavior is the function 


F(x) = y ee 


k>0 
whose behavior is sought as x — 0. The Mellin transform is 


¥2 Li) ser Gy 


iso 1255 


and the fundamental strip is (0,0). The poles to the left of the strip are at s =0 


(double) and at s = X%4.:= ae (simple). Now, as s ~ 0, 


1 sy 1 wal y 
eee Sn ( 1 ) free, 
paar pga 2 Ee fond 


and thus, as x — 0, 


1 Y 1 
F ~N 
O)~5 jog? tog? 


y r( yee, 
kAO 


log, x 


The series is a Fourier series and represents a periodic function. Since the Gamma 
function is small along the imaginary axis [49], as mentioned earlier in Section 2.9, 
the amplitude of this function is small. 
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2.13. The Mellin-Perron formula 


The following treatment is borrowed from [18]. We start from the Mellin inversion 


formula ; 
C+Ic0 


1 —s\ ¢x —s 
DA tus) = 555 de (Lau *) FY(s)x*ds, (2.5) 


where c is in the fundamental strip. Introduce the step function 


Ho(x) 1 ifxe [0,1], 
x)= 
? 0 otherwise, 


also written as Ho(x) = [0 <x < 1], together with H,,(x) = Ho(x)(1 —x)”. Then we 
get the following theorem. 


Theorem 2.13.1 Let c > 0 lie in the half-plane of absolute convergence of YAU, *- 
Then, for any m > 1, we have 


1 kym 1 retin Ay, ds 
m! yy a(1-") =a ag © et. 


l<k<n 


Form=0, 


) Se a a See 
te ee ee 
1<k<n 2 2RiSc—ico “Ue S 


This formula is obtained by setting x := 1/n, f(x) := Hm(x) in the Mellin inversion 
formula (2.5) and noticing that 


m! 


Hy,(s) = s(s+1)...(s+m) 


For m = 0, the formula has to be modified slightly by taking a principal value, since 
Ho(x) is discontinuous at x = 1. See [3] for a direct proof of this instance. 
For example, for A; = 1, Uy, =k and m = 1, we get 


n— 2ctise , as 
py (1 7) zs 7 ~ iri [. S(s)n toy 


l<k<n 


Shifting the line of integration to the left and taking the poles at s = 1 and s = 0 into 
account (note that ¢(0) = — 5) we get 


Le rare «yds 
= — ae 2) 
rae S(s)n s(s+1) ee 


This formula is the basis of some exact formule. 
We apply the Mellin-Perron machinery now to the binary sum of digits func- 
tion. Let v(k) be the number of digits 1 in the binary expansion of the integer k. 
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Furthermore, let v2(k) be the exponent of 2 in the prime decomposition of k (so, 
if k = 2'(27+1), then v2(k) = i). Then we notice that the binary representation 
looks like * * *10! (for k), and * * *01! (for k — 1). Taking differences, we see that 
v(k) —v(k—1) = 1—ve(k). Summing this on k, we see that 


v(k) =k—¥ vo(j). 


isk 
We will study the summatory function 
n(n—1) . n . . 
si):= Dv@="— ¥ n= (3) -Le-dvrl. 
k<n j<k<n j<n 
Now we compute 
v2(k) v2(2k + 1) v2 (2k) 1 1+ vo(k) 
V(s)= >: Be » (2k +1) 2 » (2k)s ~ 9s oe Es 
k>1 k>0 k>1 k>1 
7 ica S(s) 
S 
V)=6)+5V0) = Vo)= ee 
Thus 


a ge Cc 
2 2Hi Jojo 28-1 s(s+1) 
The integrand has a simple pole at s = 1, a double pole at s = 0 and simple poles at 
S=%= moe Shifting the line of integration* to R(s) = -4 and taking residues 
into account, we get 


1 
S(n) = qitlogn +nFo(log,n) —nR(n), 


where the Fourier series akin to Fy occurs as the sum of residues of the integrand at 
the imaginary poles s = 7;. The remainder term is 


1 p-at f(s) ds 
Ra) == s 
(n) ani jo 25—1" s(s+1) 


q- 


We are going to prove that R(n) = 0 whenever n is an integer. The integral converges 
since |S(-4 + it)| < |t|>/4 (cf. [49]). Using the expansion 


1 _ 
ra ae 


1 9s 92s 2 3s 


“Technically, we integrate along a rectangle with upper and lower sides passing through (2N + 1)zi/log2, 
respectively, and let N — co, Because of growth properties of the zeta function, the contribution along the 
horizontal segments vanishes. This also proves directly that the sum of residues at the complex points 
(which gives the Fourier series) converges. 
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in the integral, which is legitimate since now K(s) < 0, we find that R(n) is a sum of 


terms of the form : 
1 — 4 ties k ds 
ues anys 
bai bite S20) 44): 


and each of these terms is 0 by virtue of (2.6). 
This proves a result originally due to Delange [11]. 


Theorem 2.13.2 The sum-of-digits function S(n) satisfies 
1 
S(n) = zitlog, n+nFo(log, 7), 


where Fo(x) is representable as a Fourier series Fo(x) = Lpez fyer™™ and 


_ log, a 1 3 


fo= 3 ~ Fiog2 4 
1 ox) 2k 
te log2 %%(%% +1) for 2X log2’ is 


It is interesting to compare this derivation with the elementary and pretty argu- 
ments provided by Delange [11]: We use the notation x = |x| + {x} with integer part 
€ Z and fractional part 0 < {x} < 1 and write m in binary as (...a2a1a0)2; it is not 
hard to see that 


a= [5e|-2[5r]=f"" ([ze]-2l grea 


Therefore, with / = |log,n|, 


sw =¥ f ([zl-alare| a-ha f (enl—atedyaw 


k=0 


Introducing g(u) = |2u| —2|u| — 5 then 


+1). 
s(n) = ED sy atte) 
k=0 


= nlogyn ps) 1- {log,n} 4 n2) {logan} ye Troe ot 
2 2 io 


the last step was the change k := / —k and noticing that g(x) = 0 whenever x is an 
integer. Introducing 
h(x) = Vy 2-*g(x-2*), 
k>0 
this reads as 


__ nlogyn 
2 


iy i= ca + n2!— {logs nt p(2{log.m}—1)_ 


T 


S(n) 
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The final step is to introduce 


F(x) = SF palate, 


nlogyn 


S(n) = 


It can be shown that Fo(x) is periodic and continuous, and the Fourier coefficients 
can be computed as well. 
We want to finish the discussion of digital properties by considering the Gray 
code and the sum-of-digits function of it, again using the Mellin-Perron technique. 
The Gray code representation of the integers starts like 


+nFo(log,n). 


0, 1,11, 10,110,111, 101,100, 1100,1101,...; 


its characteristic is that the representations of n and n+ 1 differ in exactly one binary 
position, and it is constructed in a simple manner by reflections based on powers of 


two. Let y(k) be the number of 1-digits in the Gray code representation of k, and 
Ox = y(k) — y(k — 1). It is easy to see that do, = d;, and the pattern for odd values is 
55x41 = (—1)*. Thus the Dirichlet series 5(s) relative to the sequence 4; is given by 
Sy 28 (=1)* 
d(s) = = ; 
(8) = Lis Bai 2 Oke 


So the summatory function G(n) = Y,<, 7(k) of the sum-of-digits function of the 
Gray code representation can be expressed via 


n 2+i00 ds 
270i i os s(s +1) 


and the rest of the analysis is very similar to before, and leads to the explicit result 


1 
G(n) = gitlogyn +nF\ (log, n), 


with explicit Fourier coefficients of F(x). We refer for details to the fundamental 
paper [18]. 
We would like to mention the Hurwitz zeta function [49] 


1 
C(s,a):= ) ———— forO<a< lands > 1. 
X (n+a)$ 


Then 


(—1)* oe 1 1 il 1 
dL ak+ i -h (4k +1)5 Pcraey = 55(%3) 3 


This shows that 5(s) can be expressed via €(z,a). 
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2.14 Mellin-Perron formula: divide-and-conquer 
recursions 


The aim of this section, as described in [15], is to solve the divide-and-conquer re- 
cursion 


Sn =F injo| +F{nja + en, 
where é, is a given sequence. The initial conditions are chosen to be e9 =e; = fo =0. 


The tool will be a special case (m = 2) of the Mellin-Perron formula as discussed 
before. 


Lemma 2.14.1 Let wy be a sequence and W(s) = Ynys Wn/n’ its generating Dirich- 
let series. Assume that W(s) converges absolutely for R(s) > 2. Then 


n 3+ ic0 ds 
- i =¥ (n—k)wp. 
ra W(s)n +1) (n—k)we 


A265 s(s 


The recursion is now written for even and odd indices separately: 
fom = 2 fin + €2m, 
fom+1 = Smt fmt + e2m+1; 
which holds for m > 1. Taking backward differences with Vf, = fy, — fy—1 and Ven = 
€n — €n_1 Yields 
V fom = V fin + Vern, 
V fomat = Vint + Veam+1 
for m > 1. Now we take forward differences. Note that AV fin = A(fin — fin—1) = 
Sint — 2 fin + fin-1- SO 
AV fam = AV fin + AVeam, 
AV fam+1 = AVeam-+1 


for m > 1, with AV f, = fo —2f1 = e2 = AVe,. Now set w, = AVF, and its Dirichlet 
generating function W(s) = ):,>1 Wan *. From the recursion we get by summing 


AV fin AVem  W(s) AVem 
W(s) = -+AV fi + = —, 
XL (2m)° xX ms ds XL ms 
* 1 AV 
em 
Wis) = : 
(9) = Ta98 XL m: 


It is easy to check that 


n 


YV (n- WAV ik = fn —Mfi- 


k= 1 
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We assume that e, = O(n). The Mellin-Perron formula thus gives us 


n tie E(s)n> ds 
fn=fi4 2ni is 1—2-5 s(s+1)’ 


where 


The growth condition on e, ensures that this Dirichlet series converges for R(s) > 2, 
as required. 

This is an exact formula. Asymptotics can be derived from it as in previous in- 
stances, by shifting the line of integration to the left, and taking residues into account. 


Example 2.14.2 We study the “worst case of the number of comparisons in merge- 
sort,” which, according to [15] and the literature cited therein is given by our re- 
cursion, for é, =n—1 and f, =0, which implies AV f, = e2 = 1 and AVe, = 0 for 
n> 2. Then 

Ti 1 sti on ds 

n 2ni Vos 1-2-5 s(s+1) 
The residue calculations involve a double pole at s = 0 and simple poles at s = x4 = 
2nik/log2. Then one gets 


fn =nlog,n+nA(log,n) +O(V/n), 


with 
A(x) = y aero 
keZ 
ae 14 1 1 
a9=-=—->-~ and q= ork #0. 
o 2 log2 *~ log2 PAC eh 


For more details and more examples we refer to [15]. 


2.15 Rice’s method 


Rice’s method made its first appearence as Exercise 5.2.2—54 in [32]: It allows us to 


write the alternating sum 
n 4 k 
> (-1)"fx 
k=0 é 


as a contour integral 


1 (—1)"n! 
20 i 2(z—1)... (Ganyt 
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where f(z) is an analytic function that extrapolates the sequence f;, (f(k) = f,), and 
@ encircles the interval [0..n]. As was pointed out in [23], such integrals can be traced 
back to Norlund [39]. The advantage of this representation is that, by extending the 
contour of integration, asymptotic equivalents can be derived, usually by collecting 
additional residues. In many cases, the integral disappears when the contour goes 
to infinity, leading to identities. Furthermore, the alternating sum usually involves 
heavy cancellations, and is not always easy to analyze in a direct way. 

Consider the difference operator A, defined by Af, = fx41 — fg. Then it is a stan- 
dard exercise by induction to prove that 


n n 7 
A"fo=¥y ({)o" are 
k=0 
The transformation of sequences 
n n k 
ta? 8n = y @ (1) 
k=0 


is an involution, called Euler transform; if F(z) and G(z) are the ordinary generating 
functions of the sequences f, and gy, then 


Ge) =) 2 ({)(-pin= Yon iy (Je 


O0<k<n O<k n>k 
k 
z 1 Z 
= 1) = F( . 
( eae 1-z \z-1 


If f(z) and g(z) are the exponential generating functions, then 


n/n __5\k 
= S()-vta=p Se yp A =e n-2). 


7 n! 
0<k<n k>0 n—K>0 


We state the integral formula as a lemma. 


Lemma 2.15.1 Let f(z) be analytic in a domain that contains the half-line |no,°). 
Then, the differences of the sequence (f,) admit the integral representation 


y ey: (7) ye a ie z(z— THE gaa! ae oi 


k=no 


where © is a positively oriented curve that lies in the domain of analyticity of f(z), 
encircles [no,n|, and does not include any of the integers 0,1,...,no — 1. 


Proof. — This is a direct application of residue calculus: 


n! n! 


z(z—1)...(z—n) = Kk=1)...1(=1)(=2).. (kan)! 
=(-1"*(2) fe . 


Res,—; 
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The kernel in (2.7) can be expressed by Gamma functions as follows 


n! (n+ U0 (z—n)  (-1)""'P(n + 1) (—z) 


z(z—1)...(z—n) T(z+1) - T(n+1—z) 


We will deal now with rational and meromorphic functions f(z). It turns out that 
the differences of the sequence (f;,) can be expressed via collecting residues. 


Theorem 2.15.2 (Rational functions) Let f(z) be a rational function analytic on 
[29,°°) Then, except for a finite number of values of n, one has 


E ("ea ay tres, <r 


= z(z—1)...(z—n) 
where the sum is extended to all poles z of f(z)/(z(z—1)...(z—n)) not on [no,°). 


Proof. — The idea is to integrate along a large circle with radius R and use trivial 
bounds. The details are in [23]. a 


The first example that we treat is 


“EQS 


for m a positive integer. Note that for negative m, this is basically a Stirling subset 
number. In this example, f(z) = z~” is a rational function, and there is only one 
additional pole at z = 0. So, we get (compare again [23]) 


=C"E len Ew aap.0sH 
=-—(z jexp ¥ oe (==) } = “ere {EE 565) } 
=—le"Jexn{ Yo Gein \ “(e"\[Jex{ 52 ig y 

JZ jz 


5, anh (" MENS) 
Uy 42lp- +303 Ill! \ 1 2, 3 


The quantities that arise here are harmonic numbers: 
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In particular, we get 


which is a classic formula. 


Theorem 2.15.3 (Meromorphic functions) Let f(z) be a function that is analytic 
on |no,©). 

(i) Assume that f(z) is meromorphic in the whole of C and analytic on Q = 
Uj=1 Yj where the ¥; are positively oriented concentric circles whose radii tend to 
infinity. Let f(z) be of polynomial growth on Q. Then, for n large enough, 


, a= nl f(z) 
Pee) ({) aa (—1) See eo 


where the sum is extended to all poles z not on |no,). 

(ii) Assume that f(z) is meromorphic in the half-plane Q defined by Rz > d for 
some d < no. Let f(z) be of polynomial growth in the complement in Q of some 
compact set. Then, for n large enough, 


¥ -0'(2) f= (1 Res. A ont 


i )...(z—n) 
where the sum is extended to all poles z in Rz > d and not on {no,°). 


The proof can be found in [23]. 
Now we study the next example, which is about trie sums; see [32, 22, 23]. They 
originate from the solution of the divide-and-conquer recursion 


ee 
fn = An +2 ss y (1) (fk + fn—k), 
k=0 
for a given (toll) sequence a,. A prototype is the sequence 


u=¥ (;) = a. 


k=2 


It arises when we set ad, =n—1 forn > 2 and fp = f; = 0: Translating the recursion 
into an equation for the exponential generating function F(z) = Y,50 fnz"/n!, we get 


F(z) = (g-Ve+14-2eF (=). 
Setting G(z) = e*F (z) = Lns0 8nz"/n!, this means 


Ge) =z—-1+e*+26(<), 
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and so 


8n > a In 


Therefore, with F(z) = e*G(z), 


n n (— 1 Nig n n (— 1 yk 
The analysis of U,, is a direct application of Theorem 2.15.3 when taking as integra- 
tion contours large circles that go in between the poles of the function (2°~! —1)~!. 
The poles are at 

20ik 

log2’ 


each of these induces a contribution of the form 


Xe =1+ 


nk = ner log. n 


The asymptotic formula follows now from the residues at s = 0 (double pole) and 
5S = Hx, k £0 (simple poles): 


1 T 1)T(-1 
tt (An—1 1) n gy ae (n+ ) ( + Xk) 
log2 log2 £% T(n+ Xx) 


2 T 
y-1 1 n 2nik-l 
=nlog,n4 ( ) E ik-logyn 
nlogyn+n( i> — 5 + Toga De (—Xx)e +0(V/n) 


Un 


The error term O(,/7) is rather arbitrary here and could be replaced by O(n“) for 
anyO0<a<l. 
Here, we used the approximation 


n 2 
cea ar aig as rm 


which is uniform in k and follows from Stirling’s formula. 


2.16 Approximate counting 
Consider the following graph. 


y 
y 
y 
yv 
v 
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Assume that a random walk starts at state 1. Random steps are done as indicated: 
if we are in state i, the probability to advance to state i+ 1 is given by gq’, and with 
the complementary probability, we stay in state i. The question is where are we after 
nrandom steps? In [24], this is called a walk of the pure-birth type, but in the older 
literature [14], it is called approximate counting, and that is where the motivation 
comes from. The states represent a counter, and the final state k is considered to 
be an approximate count for—not n of course, but loggn, with Q = 1 We are not 
discussing the applications for that idea, but since the example is quite instructive 
and appears often, also in disguised form, and was also rediscovered several times, 
we decided to include it here. A relatively new application is in [8, 46]. 

Let p(n,k) be the probability that, starting in state 1, we end up in state k after n 
random steps. 

Flajolet [14] proves the following. 


Theorem 2.16.1 () 
k-1 —1) 5 
p(n,k) =)" oud 


= k-t\n 
TCC ae ee 


The following derivation is from [36]. We obtain by a direct translation from the 
graph the recursion 


p(n,k) = 'p(n-1,k-1) + (1—-¢")p(@—-1,4),  p(0,1)=1. 


We will use a bivariate generating function. If we set 


co 00 


F(z,u):= VY z"u‘p(n,h), 
n=0k=0 


we derive 
F(z,u) —u = zuF (z,qu) +2F (z,u) —2F(z,qu), 


or 


Iterating, this gives 


u ua) ug  z(u—1)z(qu—1) 5 
1—z 1—z 1l-z l1-z (l-z 
z(u—1) z(qu—1) z(q’u—1) 3, 


+412, deg (eget 7 (2.8) 


(—1)/2/(u;q) juq! 


=) (l—z)ftl 


F(z,u) = 


This expression was derived in [43], using a transformation formula due to Heine. 
Now we have several ways of computing [z”u“|F (z, u). We write 


yy (Us q)e0 
(u;q)j —" (uq/;q)oo" 
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and with Euler’s partition identity (see Section 2.4), we have 


 (-nig) 3 
t=0 (q)1(Q)k-1-1 j=0 
I (-nigl) 


- Sr ie a 6 ee 
c= Gass” q Is 


1)ighYigi[z"-J =z) FY 


which is exactly Flajolet’s formula. 
There is a second expression (given by Charalambides [7]), 


(5) 
nk a? jr (Di (2 = 


This is equivalent to Flajolet’s formula for p(n — 1,k); we give an independent 
proof of this fact. 


“ey a piety  ED® inn 
LO aap 8 aaa 
Let us consider the generating function 

S= ¥ x‘p(n—- 1,k) 
k>0 
= k “ (—1)/g) k= jyn 
ee a Gi@es oe 
_y (2 ae he ae (—1)iq@) (kl 
=BU JV LOL Gi@uns 
y(n i (—1)/q) 1 (k—j)l 
“BV L ob Ga! 


l| 
Ms: 


(xq'sq)o 5 (Di 
1 
(xq!5q)oo 


(-1)'(@sq)r. 


r 
rosy 


(X3.q) oo 


I 
Ms: 


r 
°o 


I 
Ms 


r 
°o 
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On the other hand, let us consider the generating function 


T = Vx'pc(n,k) = Ya) ig wl) 


a "Ese “a 


T 
m. ~ 
Ms 


T 
rosy 


lI 
Ms= 
aS 
Qs 
SS 
= 
tone 
| 
a 
a 


T 
oO 


ll 
Ms 
cs 
= 
i 
~~" 
= 
Se 


T 
oO 


So S = T, which ends the proof. 
A third expression for Flajolet’s formula consists in using a g-binomial in (2.8) 
to extract [u—']. First, 


(Fea) = EG eer Na) 
(=Iid@l yl i es 
7% apr Mee | ( 1)* a 


perk) EE Gaya te! | Coe 


=)" Wei), Ge 


j=k-1 


a 


For the asymptotics of the expected value C, and the variance, Flajolet [14] first 
approximated the probabilities p(n,k) using real analysis, and then continued with 
the approximate values, using the Mellin transform. This provides additional infor- 
mation about the probability distribution in the limit. To compute C,,, there is, how- 
ever a more direct way, starting from the generating function F(z, u), differentiating 
it w.r.t. u, followed by u := 1. It should be noted that this operation, applied to (u;q) ; 
for j > 1, simply results in zero. Therefore 


0 
Ch —_ Ee") 5 Fe) 


u=1 
= [2"] y (—1)/z/(q) j-1¢7 4 oe 


jel gre 
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The asymptotic evaluation of the sum is a typical application of Rice’s method, see 
Section 2.15. We extend (q) ;-1q/ by 


(Q)co vie 
(1 — 4°) (q* eo 
Then we have to compute residues of 


(—1)"n! (q)eo i 
2(z—1)...(z—n) (1-42) (4 )eo 
at z = 0 (double pole) and at z = 7, (simple poles for k 4 0); we write 7, = 27ik/L, 


with L = logQ and Q = 1/q. The double pole requires more work, since all the 
factors have to be expanded to two terms. We collect the expansions: 


—1)"n! 1 1 1 E 
(—1)"n aay ie (14 2) 
z(z—1)...(z—n) z 1-g Lz 2 
(q) oo 
~ 1-alLz, Gg ~1-Lz, 
(q®) oo 


with a = Lei 


oat Using the asymptotics for H,, and adding the extra term 1, we 
get loggn+ ue + 5 — a. The residue at 7, is simpler: 
1T(n+1)l(-x%,) 1 


~ —nkD ; 
L Tint+1—-x) ik (—Xx) 


Altogether we have a theorem, first proved by Flajolet [14]. 
Theorem 2.16.2 The following equality holds: 


= ae! -1/2 
Cn = logon + 7 +5 —% + d(loggn) + O(n Py. 


where the (small) periodic function 6(x) is given by its Fourier expansion 


1 Tikx 
5(x)=—-F Yr(-m)er"™. 
k#0 


For more results on the subject we refer to the original papers and [24]. 


2.17 Singularity analysis of generating functions 


Our goal is to say something about [z"] f(z), using information about the singularities 
of f(z). Let us start with the simple case of a rational function f(z). Using partial 
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fraction decomposition, it can be written as a polynomial plus a finite number of 


terms of the form ri 


(—z/py 


for positive integers k. Now, the polynomial part can only influence a finite number 
of terms, is thus irrelevant for asymptotics, and 


n A —n n+k—1 —n cs 
eaaape er ( n ) ~ap Cz 


We see that the singularities p that are closest to the origin are poles in this in- 
stance. They produce the largest exponential growth p~”. For example, let f(z) = 
aay Then there are two poles at +1 of smallest modulus 1, and one at 3, 


which will produce an exponentially small error term. We find the local expansions 


f~s 40, f@~Fz & 4-H), 


whence the asymptotic formula 
[z"| f(z) =6+3(-1)"+0(3-"). 


Now this instance was extremely simple, but we can deduce an important principle: 
The location of the singularity is responsible for the exponential growth, and a pole 
of order k produces a (leading) term of order n‘~!. 

Now let us consider Catalan numbers, given by 


Oy (72. 


2z ott l n 


Stirling’s formula gives us 


1 /2n 4” 
n+1\n JVan3/2- 
Can we see this directly from the generating function f(z)? Well, the singularity is 
at z= i and the local expansion is 


f(2) =2-2V1 4a 4, 
and 
—24” 7 qn 
Pps? Vani? 
The rest of this section is devoted to the study of why and how this works. The main 


references are the highly cited article [19] and of course [24]. 
For a precise notion of exponential growth, we write 


[2"] (-2V1 — 4z) 


An &IK” if and only if limsup lan|!/" —K 


(read a, is of exponential growth K”). 
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Theorem 2.17.1 (Exponential growth formula) /f f(z) is analytic at 0 and R is the 
modulus of the singularity nearest to the origin, then the coefficient fn = |z"|f(z) 


satisfies 
1\n 
fura(R) 


This allows us to deal with meromorphic functions (the only singularities are 
poles). We can consider all poles nearest to the origin; there can only be a finite 
number of them, usually it is just one pole. Then one can subtract these poles (the 
principal parts). The resulting function has a larger radius of convergence, and thus 
what remains is exponentially small compared to the contribution from the dominant 
poles. Let us consider the example 


1 


R(z) = —— 
G5 


which is the exponential generating function of surjections. The poles are the solu- 
tions of e* = 2, or the points 7, = log2 + 27ik, k € Z. The closest pole to the origin 


is at log2, so 
1 n 
[z"]R(z) >< Gea) 


But we can write 
R(z) = : : + S(z) 
BO De loge 


and S(z) has radius of convergence |log2 + 27i| = 6.321302922. Consequently 


1 1 


[2"]R(z) = 5 


3 toga +0(6"). 


In general, if p is a dominant pole of order k, there will be a contribution p(n)p~", 
where the polynomial p(n) has degree k — 1. 

The process can be iterated by including more poles, and obtaining a smaller 
(exponential) error term. In a variety of cases, the formal process that includes all the 
poles will lead to a series expansion that is exact and asymptotic. In general, there is a 
theorem due to Mittag-Leffler that explains how a series expansion can be obtained. 
For asymptotic purposes, collecting the contributions from dominant poles will be 
sufficient. 

The process called singularity analysis of generating functions is about transfer 
theorems, from information of the generating function around the singularity to the 
asymptotics of the coefficients. Sufficient conditions will be provided for the impli- 
cations 


(T1) f(z) = O(g(z)) =>  fnr=O(n), 
(T2) f(z) = 0(g(z)) = = fn=o(n), 
(T3) f(z) ~ gz) => fn 8n- 


We will also speak about O-transfers, o-transfers, and ~-transfers. O-transfers are 
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the most basic; refinements usually lead to o-transfers, and ~-transfers follow from 
these, since 


f(z) ~g(z)_ is equivalentto f(z) = g(z) + o(g(z)). 


Basic transfer. We know from basic principles that 


n 


ewi-a-e= ("FE") Pn +a) 


~ T(a@)P(n+1) 
for a ¢ {0,—1,—2,...}; from Stirling’s formula this gives us 


a-l 


a(a—1) 


a(a—1)(a@—2)(3a—-1) 
2n mh 


24n2 


(Cla aay 


The full expansion is given in [19]. 
Without loss of generality, when analyzing a singularity at p, we can assume that 
it is as 1, because of the simple transformation g(z) := f(z/p), and 


[e"]e(z) =p “"/F(2)- 
From a technical point of view, the following domains are important: 
A(g,n) = tz] kel $1 +0, larg(z—1)| 2 9}, 


where we take 7 > 0 and 0 < @ < 2/2. This domain has the form of an indented disk 
as shown in Figure 2.4. 


[i+ 


Figure 2.4 
Domain A(@,1). 


Extraction of coefficients is then done using Cauchy’s integral formula, with a 
contour that stays as close to the boundary of the domain A(@,1) as possible. See 
Figure 2.5 for an illustration. We must refer to the original paper for details. 
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Figure 2.5 
A typical path of integration. 


Theorem 2.17.2 Assume that, with the sole exception of the singularity z = 1, f(z) 
is analytic in the domain A = A(¢,n), where n > 0 and0 < @ < 1/2. Assume further 
that as z tends to 1 in A, 


f(z) = O(|1-z|"). 


for some real number a. Then the n-th Taylor coefficient of f(z) satisfies 


fa = (2"|f(z) = O(n™""). 
Corollary 2.17.3 Assume that f(z) is analytic in A\ {1}, and that as z tends to 1 in 
A 
f(z) =o({l-2-%), 


Then, as n — ©, 
fr= o(n*"). 
Corollary 2.17.4 Assume that f(z) is analytic in A\ {1}, and that as z tends to 1 in 
A, 
feeKIa=w 
for a ¢ {0,—1, —2,...}. Then, asn + , 


This settles the instance about the Catalan numbers from before. Since V1 — 4z 
can be extended to the complex plane except for a cut from Fi to infinity on the real 
axis, this also includes a fortiori a A-domain. 

Now we also allow logarithmic factors. In other words we consider 


f(=( *(tee(4))" 


The factor 1 /z is introduced for convenience: Since log 1/(1 —z) =z+0O(z’), divid- 
ing by z lets this expansion start with 1, and then raising it to the power B results in 


236 Handbook of Enumerative Combinatorics 


a power series expansion. The expansion is given in [24]: 


C C2 
logn log? n 


z (logn)? [! + 


with 


Note that this expansion comprises terms that have the factor n®—!. Terms including 
~? go into the remainder term. 
Here is an example: 


A 1 1 y+2log2 1 )) 
z ——— => T é 
Vi-ztlog;, Vanlogn logn log?n 


It is also possible to introduce an additional factor of the form (log log n)°; we 
are not citing the details here. 

The instance of @ being a negative integer had to be excluded before, since it 
resulted in a polynomial. With the presence of logarithmic factors, however, this still 
makes sense. Here is one particular expansion: (n > m > 0) 


a) ae Goat 


[z”](1 —z) "log 7 


The following proof is quite instructive: Consider 


m! 1 


z2(z—1)...(z—m)z—n 


f(z) = 


and perform partial fraction decomposition: 


f(zj= y et rt + m! 1 


Fars n(n—1)...(a—m)z—n- 


Now multiply this by z and let z > o. The result is 


=) (7) a ee cee ey] 


2.18 Longest runs in words 


We consider a binary alphabet {0,1}, and are interested in the length of the longest 
run of ones in a random word of length n. Much more general scenarios are dis- 
cussed in [24]. For a given parameter k, we want to find the generating functions 
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of words where all runs of ones have length < k; call it W<*(z). There is a natural 
decomposition of the words with this property: 


1<«(01<*)", 
where 1<* = {¢,1,11,...,1‘~!}. From this, 


1-2 1 1—z 

<k _ _— 

We) ae a ea 
TG 


The first step is the location of the dominant pole p;, which we expect to be close to 
5. Set Q(z) = 1—2z+2**!. As z traverses the circle |z| = 1 in the complex plane, 
the value of Q;(z) winds around the origin exactly once, hence the polynomial Q, 
has exactly one root in |z| < 1. We call this root p, = 5 + &. It satisfies the equation 


One can start with a crude bound & = O(;) and plug this in: 


Bee ao 
go ESB OND k 


This leads to the approximation 


1: 1 k 
Pk= 5 ses + O(a); 


which is enough for practical purposes. We can do better, however, using the La- 
grange inversion formula. We write y = z— 5 and x = 5 then 


y =x(5 ty) = x®(y). 


Z: 
ne 1 1 \nle+Ht) 1 n(k+1)\ 1 
NS nT pf = net: Renee nk 
iy =p I(5+y) =o ) aa 
and | 
= 1 (n(k+1 1} 4, 
r= D5 n—1 ) sar 
Plugging in the special value for x, we get 
1 1 (n(k+1) 1 (re Sew 
a= 5 +0; n—1 ) mem = pt geet 


The iterative process to get better and better approximations for p,; is called boot- 
strapping and appears as early as in [33]. The next step is to expand W<*(z) around 
the dominant pole. We have 


A 


w<K(z Ky ; 
«) 1—z/px 
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and ; k k 
: 1-z 1—p; 1—p; 
A= lim(1—z = = : 
At! ~2/P) Oey = XpeO"(pn) ~ pl — (e+ Deh) 


Therefore A 

1 py —n-1 
5 (kL ok Pe 
2—(k+ lof 
The error comes from the fact that the remaining poles are larger than | in absolute 


value. This derivation, as it stands, works for k fixed and n — ©, but it is not too hard 
to extend the range of validity of it. In particular, one replaces A by the easier 5 and 


“w<F(z) = +0(1). 


lvl 1] —n—-1 " 1 n+l n—nsoktl 
PW Os tpn) age) 


when k is close to log,n where the main contribution comes from. We must refer to 
[24] for this series of approximations. 

It is interesting to sketch the enumeration problem that Knuth [33] encountered 
in his carry propagation problem. Words of length n over the alphabet {0,1,2} are 
studied where letter 1 appears with probability > and 0 and 2 with probability i 
each. For given k, the (contiguous) substring 1*2 is forbidden. The allowed words 
may be described as 


((a<#2)"1"0) "(1<#2) "1" 
Translating that into a generating function (0+> z/4, 11> z/2, 2+ z/4), we get 
1 
1—z+4(g)e1 


The rest of the analysis is very similar to the previous discussion; the dominant pole 
is here close to 1. 


2.19 Inversions in permutations and pumping moments 


We consider permutations 7 = pip2...pn of {1,2,...,n} (written in one-line nota- 
tion) and assume that all n! of them are equally likely. An inversion in 7 is a pair i < j 
with p; > pj, and I(7) is the total number of inversions of 7. There is a convenient 
way to study the statistics of this parameter, namely the inversion table bj b2...Dn. 
The meaning is that b; counts the number of elements larger than i that stand to the 
left of i. Clearly, I(a) = b, +---+b,. We have the natural restrictions 0 < bj <n—i, 
since there are only m — i elements larger than i altogether. The cute observation is 
that, given an inversion table, the permutation itself can be reconstructed: The value 
b, tells us that number | will be in position b; + 1, then 5 tells us the position of 
number 2 (b2 slots must be left open for later numbers), and so on. There is thus a 
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bijection between permutations and inversion tables; the latter ones are easier to han- 
dle when it comes to generating functions, because of independence of the entries. If 
we consider the product 


BS 1—q/ 
(1+q)(ltq+q)--(lt+gt--+9"")=]] 
j=1 


then the coefficient of g* in it is the number of permutations of n elements with k 
inversions. Now let 


1—g! 1—¢" 
Fo) = oq gy = ac ay 


be the probability generating function of the parameter /(7). Our goal is to get infor- 
mation about the sth factorial moment, ie n(Q) Late We form a bivariate generating 
function 

H(z,q) =) 2"F,(q). 


n>0 
The recurrence relation then translates into 


(1 ~@) 2 H(e,9) =H(z,q)—qH(zq,q),  -A(z,1) = = 


It is impossible to solve this functional differential equation explicitly, but it is pos- 
sible to derive information about 


@ ad 
85(Z) = 2 Geka Da = Gg Mga 


n>0 
by differentiating the functional equation several times, and expressing g,(z) by the 
known values g;(z) with i < s. This procedure is nicknamed pumping moments in 


[24]; the following computations are taken from [29]. 
With a bit of patience, this program results in 


1 1 


gs(z)=hs(z), gol) =—— 


‘4 
8s(2) — 7 es 


where 
Tf VN ie 8 1 SEE TNS ey ; 
h = J zt f+ ——_____ (4) az 
= T LG )ee (N02) bail j Jr 


Note that /,(z) is a “known” function since it involes only g;’s and its derivatives that 
were already computed. This differential equation is easy to solve: 


85(z) = fon —t)dt. 
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By inspection, one “sees” (and then proves by induction) that 


_- _ -(2s)! _ ¢s(2s—1)! 
pa) 48(1 —z)s+1 ( —z)?s 


+ lower order terms. 


hs(z) 


The constants cs satisfy 


s s(1—4s) 


ith cy =0. 
10521). 3Ossnaa ee 


Cs = Cs—] 


It is easy to prove that 

(4s +5) 
a 
This leads by direct translation to the formula 


Cs = 


d’ 


ae (q) lest = 


1 s(2s—11 = 
°F a 25 4 ( ) ps 1 


4s” 9.45 


+ O(n5~?), 


For s = 1, we get the expected value ae, which is exact. 
If one applies the pumping moments method to a suitably shifted random vari- 
able, one can derive limiting distribution results. See [24] for more information. 


Area under Dyck paths. In this example, we consider Dyck paths (counted by 
Catalan numbers) and their area; if the path is xox, ...x2, with x9 = x2, = 0, x; > 0, 
Xj; —Xj41 = +1, then the area is defined to be xo +x; +--+ +x2n. Let F(z,q) be the 
generating function according to half-length and area. Mapping a; +> giz, b jt qi, 
c;++ 0, the continued fraction theorem (Theorem 2.7.3, Section 2.7) leads directly to 


1 1 
zq 1 —zqF (zq,q) 


F(z,q)= 


It is natural to set F (z,q) = A(z)/B(z), then 


A(z) 1 7 B(zq) 


Biz) 1—zqSle— B(zq) — z4gA(za) 


Comparing numerator and denominator, 
A(z) =B(zq), — B(z) = 1—zqA(zq) = 1 — zqB(zq"’). 
Setting B(z) = Y,,bnz”, this leads to 


bn =Qrbn-Gr 'dn-1, bo = 1. 
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This recurrence can be iterated, with the result 


Defining 
“7 wid 
E(z,q) = + oe. 


n>0 (9)n 


we may express the sought generating function as 


E(zq.9) 
F(z,q)= 
E(z,q) 
Now we look at the generating functions L,(z) = oF (z,q) Loe they are (apart from 


normalization) the generating functions of the rth factorial moments. Pumping the 
moments can now be done as follows: First we rewrite the functional equation as 
F(z,q) = 14+ 2F (z,q)F (zq,q). Then this will be differentiated r times according to 
the Leibniz rule, resulting in 


we) =2b (“Jad Leu Pst. 
k=0 


j=0 
One sees by inspection that 


K, (37S 
te(2) = G gaara t OU 42) ae oy 
The constants K, follow a recurrence, which cannot be solved explicitly, but this 
information is sufficient to characterize its probability distribution [24, 47]. This is 
also true for our previous (simpler) example of inversions, which leads to the normal 
distribution. 
Many similar problems exist in the literature for which such an approach works. 


2.20 The tree function 


The objects we consider are labeled rooted non-planar trees. There is a symbolic 
description of this family: 


TF = ZL xSEV(T). 


This is a recursive description, which uses the labeled product that conveniently does 
the relabeling for us once we use exponential generating functions: 


T(z) =ze™), 
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Reading off coefficients is a typical application of the Lagrange inversion formula: 


1 1 yn’! 
NT = {pr} Tn . 
ke" = le n(n—1)! 


Therefore the number of labeled rooted non-planar trees with n nodes is given by 
al[2"|T =n"), 

This is a famous formula, and many direct proofs exist for it. We refer to [24, 31] 
for pointers to the literature. 

For comparison, let us also study the unlabeled counterparts, defined by 


A = &-MSET(&), 


which leads to 


A(z) = zexp (A) + 5A?) +542) +3 


this equation for the ordinary generating function A(z) = Y,> 1 anz" was first derived 
by Polya. There is an equivalent formula, 
z 


AGG ra leet a bea Nerd es 


which is also implicit, but allows us to compute the numbers a, in a recursive fashion. 
The formula follows directly from the combinatorial description: What follows the 
root is a multiset of already existing trees of all possible sizes. 

The implicitly defined function y = ze” is known as the tree function. This is an 
important function that, although implicitly defined, we should add to our arsenal of 
known functions. Whenever one sees quantities like n”, one should think about this 
tree function. We know that 


and then 


Many similar quantities may be expressed via the tree function. A celebrated example 
is Ramanujan’s Q-function [17]: 


n—1  (n—1)(n—2) 


=14 pees 
O(n) = 1474 2 
It follows immediately that 
n” yn’! n'-2 
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Now consider log =C) and its coefficients. This is a typical application of the third 


version of the Lagrange inversion formula, with g(y) = log = We compute 


1 1 1 1 Pate 
m4 Sarah yn = yyn—1 ese Le 
[z"] iss So le ey lege poe 


so we see that log 74, is essentially the generating function of Ramanujan’s Q- 


function: 
1 n—1 


lo = nz". 
Saerey d 7 Gln) 

Next, we look again at the Lagrange inversion scenario, y = z®(y). In combi- 
natorial contexts, it is natural to assume that B(y) is given as a power series with 
non-negative coefficients ®(y) = Y, dyy* and that @ > 0. To avoid trivialities, we 
exclude the linear function ®(y) = $9 + @1y. This describes planar trees, and the 
number @, can be interpreted as a weight when branching with k successors occurs; 
in particular, if @, = 0, k-way branching is not allowed. The family of such trees is 
often called simply generated family of trees, introduced by Meir and Moon [38]. 
The tree function y = ze” is the instance B(y) = e”. The radius of convergence of y(z) 
(and thus the exponential growth of the coefficients) can be determined by a theorem 
that we cite from [24]. 


Theorem 2.20.1 Let ® be a function analytic at 0, having non-negative Taylor coef- 
ficients, and such that ®(0) 4 0. Let R < © be the radius of convergence of the series 
representing ® at 0. Under the condition 


there exists a unique solution Tt € (0,R) of the characteristic equation 


T®'(T) 


®(T) 


Then, the formal solution y(z) of the equation y(z) = z®(y(z)) is analytic at 0 and its 
coefficients satisfy the exponential growth formula: 


T 1 
n —n h, = ? 
[z"]y(z) > p where p= an) ~ OD) 
T®!(T) 


We just give the remark that the equation ey 1 follows from the implicit 
function theorem. 

Now we apply this to the tree function. It is plain to see that tT = | and hence 
p= i. We want to expand y(z) around its singularity p = i. In order to do so, it 
is easier to look at z = z(y) = ye’, expand this around t = 1, and then invert the 
expansion. We easily get 
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Note that the linear term is not present, which will lead us to a singularity of the 
square-root type. We get 


(l—y)? =2(1—-ez)+--- = y=1-V2V1—ez4+ --. 


The conditions of singularity analysis are satisfied, since the function y(z) is analytic 
except for a cut from : to co on the real axis. Therefore 


n 
n é 


: 1 
[z ly(z) is) ~via ane = Vanni? 


n- 1 


n! 


n 
n!~ V2an (=) : 
e 


But we know the exact answer [z”]y(z) = . Hence 


Note that in principle as many terms as desired could be obtained. Singularity analy- 
sis has proved Stirling’s formula via the tree function! 


2.21. The saddle point method 


Our main source here is Odlyzko’s chapter [40]. 

The saddle point method is the most useful method for obtaining asymptotic 
information about rapidly growing functions. It is based on the freedom to shift con- 
tours of integration when estimating integrals of analytic functions. We assume that 
f(z) is analytic in |z| <.R < oo, We will also make the assumption that for some Ro, 
if Ro <r <R, then 


max|f(2)| = f(r). 
Iz|=r 

This assumption is clearly satisfied by all functions with real non-negative coeffi- 
cients, which are the most common ones in combinatorial enumeration. We will also 
suppose that |z| = r is the unique point with |z| = r where the maximum value is as- 
sumed. The first step in estimating [z"|f(z) by the saddle point method is to find the 
saddle point. Under our assumptions, that will be a point r € (Ro,R) that minimizes 
r—"f(r). The minimizing r = ro will usually be unique, at least for large n. Cauchy’s 
integral formula is then applied with the contour |z| = 79. The reason for this choice 
is that for many functions, on this contour the integrand is large only near z = ro, the 
contributions from the region near z = ro do not cancel each other, and remaining 
regions contribute little. By Cauchy’s theorem, any simple closed contour enclos- 
ing the origin gives the correct answer. However, on most of them the integrand is 
large, and there is so much cancellation that it is hard to derive any estimates. The 
circle going through the saddle point, on the other hand, yields an integral that can 
be controlled well. 
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Example 2.21.1 (Stirling’s formula) We compute the reciprocal of n\ as [z"|e*. The 
saddle point is that real r that minimizes r~"e", which is r = n. Consider the contour 
\z| =n, and set z=nexp(i0), -t@ <0 <a. Then 


: 1 dz Ls «if i0 . 
nN) 3% Zz n L 
[Z"]e* = Or [ue ful oe fo exp(ne’’ —niO)d@. 


Since |exp(z)| = exp(N(z)), the absolute value of the integrand is n-" exp(ncos 8), 
which is maximized for 8 = 0. Now 


: 2 
e® —cos@ +isin@ = 1— 5 +18 +0(16)), 


so for any 9 € (0;7), 


6 a5 8 n@~ 3 
/ n"exp(ne’ — nie) do = | n"exp(n—— + 0(n/6| )) dé. 
—0o — 6 2 


Note the cancellation of the ni@ term. We select 09 = n—2/>, so that n\6|> < n71/5, 
and therefore 


exp (n- ee +o(nl0)*)) = exp ((n- ) (1 +o(n-"*))) 


Hence 
8 ; 8% 2 
/ n"exp(ne® — nid)d@ = (1 + O(n) nner f exp(— 22 )d@. 
—0o —69 


But 


= (2a/n)'? + O(exp(—n'/°/2)), 


4 . 
/ ; n-"exp(ne® —nid)d@ = (1+ 0(n-!/9))n"e"(2n/n)!/?. 


On the other hand, for 0) < |@| < 1, 
9% 


cos 8 < cos 0 = 1— > +0(65) 


SO 
nceos@ <n—n'/>/24+0(n>/>) 


and therefore for large n 


TU - 
i, n-"exp(ne — nid) d0| <n" exp(n—n!/> /2). 
% 
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Combining all these estimates we find that 

— = [e"Je* = (1+ O(n-/9))(2n/n)'/?n-"e", 

which is a weak form of Stirling’s approximation. 


Let us summarize: We decompose the contour ¢ = @ U@(), where @) (the 
“central part’) contains the saddle point (or passes very near to it) and 6) is formed 
of the two remaining “tails.” This splitting has to be determined in each case in ac- 
cordance with the growth of the integrand. The basic principle rests on two major 
conditions: the contributions of the two tails should be asymptotically negligible; in 
the central region, the quantity f(z) in the integrand (which is written as exp(f(z))) 
should be asymptotically well approximated by a quadratic function. Under these 
conditions, the integral is asymptotically equivalent to an incomplete Gaussian in- 
tegral. It then suffices to verify that tails can be completed back, introducing only 
negligible error terms. By this sequence of steps, the original integral is asymptoti- 
cally reduced to a complete Gaussian integral, which evaluates in closed form. 

Let us consider another example where the outcome is less predictable. It goes 
back to van Lint [35], but compare with [37] for further developments. 


Representations of 0 as a weighted sum. Let A(N) be the number of solutions of 


the equation 
N 


yy ek=0, where & € {0,1}. 
=—N 


Now 
Nv. ye 
A(N) = [2° ( ( 
(N) el TT ¢ +2) = =f IL +2) 


In order to find the saddle point, it is convenient to take the logarithm of the integrand, 
differentiate, and solve the equation 


The solution (approximate saddle point) is close to one; if one wants a closer ap- 
proximation, one can use bootstrapping, as explained earlier. We thus choose the 
unit circle as the path of integration: 


ae 


ae []eos* kx dx. 


Now we sketch how to choose appropriate ranges and approximations. For the range 
W <x< ae the integrand is exponentially small, and can be ignored. For 0 <x < 


N~‘4/3, the Gaussian approximation is valid: 


N 
[ cos? kx = exp ( Pies Hen z +o(n-"/3)); 
k=1 
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the integral over the remaining range N BE atece >y is not neglible, but smaller 
than the contribution from the central range. All error terms can be made explicit, 
and since 


ee (VN +1)( ) 
| A NOEL) 


1 
exp dx~ 5 3a)'?N?, 


there is the result ae 
A(N) ~ (=) ? N+ y-3/2, 
1 


2.22 Hwang’s quasi-power theorem 


Here, we want to discuss a method that turns out to be very useful in combinatorial 
contexts to prove Gaussian limit laws. This section is by no means a full description 
of the rich interplay between combinatorics and probability theory. The source of our 
short treatment is once again [24]. 

A real random variable Y is specified by its distribution function, 


P{Y <x} =F(x). 


It is said to be continuous if F(x) is continuous. In that case, F (x) has no jump, and 
there is no single value in the range of Y that carries a non-zero probability mass. 
If in addition F (x) is differentiable, the random variable Y is said to have a density, 
g(x) = F’(x), so that 


x 
P{Y <x}= [ g(x)dx. 
A particularly important case is the standard Gaussian or normal -¥ (0, 1) distribution 


function, 


P(x —w"/2 dy: 


«= fe 


the corresponding density is 


Let Y be a continuous random variable with distribution function Fy (x). A se- 
quence of random variables Y,, with distribution functions Fy, (x) is said to converge 
in distribution to Y if, pointwise, for each x, 


jim Fy, (x) = Fy (x). 


In that case, one writes Y, = Y and Fy, => Fy. 
For the readers’ convenience, we cite a classic theorem. 
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Theorem 2.22.1 (Basic central limit theorem) Let 7; be independent random vari- 
ables supported by R with a common distribution of (finite) mean [ and (finite) stan- 
dard deviation 0. Let Sy, := T, +-+--+T,. Then the standardized sum S*, converges in 
distribution to the standard normal distribution, 


Sp — pn 
oV/n 
Short proofs use the concept of characteristic functions, which we do not discuss 

here. 


A particularly simple application is when T = T; takes the values 0 and 1, both 
with probability 5. Then we can consider the probability generating function 


1 uy” 
Pa(u) = (5 an =) ) 
so that [u*]p,(u) =2~"({) is the probability that S,, = k. We see here that the proba- 
bility generating function is a large power of a fixed function, here He 

We will see that it suffices that the probability generating function of a combi- 
natorial parameter behaves nearly like a large power of a fixed function to ensure 
convergence to a Gaussian limit. This is the quasi-powers framework, a concept that 
is largely due to Hwang [28]. 

Before we state a theorem, we discuss the cycle distribution. Assume that we 
have a random permutation of n elements, and the random variable X,, counts the 
number of cycles. Since a permutation can be seen as a set of cycles, it is easy to 
write down the relevant generating functions: 


Si = =>. (0,1). 


P=SE(CYC(#)) = > P(z,u) =exp (ulog f=7 


\=(-2)™. 
nk 


So, n![z"u*]P(z,u) is the probability that a random permutation of n elements has k 
cycles. Extracting the coefficient of z”, we find the probability generating function 


Pn(u) related to X,,: 
— (ntu-l\ Tlutn) . 
pate) = (nn) = ae 


near u = 1: 


ni! ell logn 
pa) = (140(4)) =I 14 0(4)), 


Thus, as n —> 0, the probability generating function p,(u) approximately equals a 
large power of e“~!, taken with exponent logn and multiplied by the fixed function, 
T'(u)~!. This is enough to ensure a Gaussian limit law, as we will see. 

The following notations will be convenient: Given a function f(u) analytic at 
u = | and assumed to satisfy f(1) 40, we set 


_ fl) _ £0), £0) a, 
F(1) IO FD NFO” 


mean(f) 


var(f) 
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Theorem 2.22.2 (Hwang’s quasi-power theorem) Let the X,, be non-negative dis- 
crete random variables (supported by No), with probability generating functions 
Pn(u). Assume that, uniformly in a fixed complex neighborhood of u = 1, for se- 
quences Bn, Ky, — °, there holds 


pala) =A@)B)®(140(=) 


Kn 


where A(u), B(u) are analytic at u = 1 and A(1) = B(1) = 1. Assume finally that 
B(u) satisfies the so-called “variability condition,” 


var(B(u)) = B"(1) +B"(1) — (B'(1))? £0. 


Under these conditions, the mean and variance of Xy, satisfy 


Un = EX, = Bymean(B(u)) + mean(A(u)) + O(k;,'), 
o? = VX, = B,var(B(u)) + var(A(u)) + O(K, !). 


The distribution of Xp is, after standardization, asymptotically Gaussian, and the 
speed of convergence to the Gaussian limit is O(K;! po"), That is, 


pf a <x} = (x) +0(+ TE) 


This theorem is a direct application of a lemma, also due to Hwang, that applies 
more generally to arbitrary discrete or continuous distributions. 
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3.1 Introduction 


The focus of this chapter is the frequent appearance of the normal distribution in 
the context of combinatorial enumeration. The notion of asymptotic normality is 
defined, and four methods for establishing its presence are stated and illustrated. The 
connection of asymptotic normality with other methods of approximate enumeration 
is briefly explored. 
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3.2 The normal distribution 


A random variable X is said to be normally distributed when 


1 * 2 
Prob{X <x} = —— / et Pat. 
aS ag a 
We shall denote the function on the right by ./ (x). Even though their work is sepa- 
rated by about one hundred years [24, page 55], de Moivre and Laplace share credit 
for the following fundamental theorem concerning independent Bernoulli trials. 


Theorem 3.2.1 Let 0 < p< 1, let Xj, 1 <i<n, be asequence of independent random 
variables each of which takes the value 1 with probability p and the value O with 
probability q = 1— p, and let S, = ¥-"_, Xj. Then, for any fixed x, 


Prob(Sn <np+x,/npq) > W(x), asn—-o, 


We remark that np and npq are the mean and variance, respectively, of S,, and 
that the assertion of the theorem may be written explicitly in terms of binomial coef- 
ficients as 

peg ew 3.1 
,jP¢ > (x), asn— oo, (3.1) 
k<np+x,/npq 


Many textbooks on probability and statistics introduce the normal distribution as an 
approximation to the binomial distribution, as in the previous equation. Theorem 
3.2.1 may be proved by estimating the individual terms via Stirling’s formula and 
approximating the sum with an integral. Exactly this approach is followed in Section 
VIL.2 of Feller [17]. 

The normal distribution is also known as the Gaussian distribution. Having 
achieved fame with his calculation of the orbit of asteroid Ceres (discovered in 
1801), Gauss later published a treatise on the subject of celestial orbits. In this work 
he propounded the method of least squares, and provided theoretical justification 
by assuming that observational measurements are normally distributed about their 
mean. 

The focus of this article is the frequent appearance of the normal distribution in 
the context of combinatorial enumeration. The typical situation is that for each pos- 
itive integer n we have a family of combinatorial objects, such as all permutations 
of the integers [n] = {1,2,...,m}. It is desired to count these with respect to a pa- 
rameter, for instance how many permutations of [n] have k cycles ? This leads to a 
doubly-indexed array of counts, a(n,k) say. When the objects are given the uniform 
probability, the parameter can be viewed as a random variable X,,. Denoting the mean 
of X, as [, and the variance as o,, it is often found that the probability of the event 
Xn < Un +xO, approaches -V (x) as n — oo, In this situation, we say the counts a(n,k) 
are asymptotically normal. We repeat the definition. 
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Definition 3.2.2 Let a(n,k) be a doubly-indexed sequence of nonnegative numbers. 
We say that the sequence a(n,k) is asymptotically normal with mean 1, and vari- 
ance 6? provided that for the normalized probabilities 


a(n,k) 


me Falak) 


we have, for each x, 


yy p(n, k) + V(x), asn— oe, 


K<Untxon, 


In the next few sections we give, with illustrative examples, four methods of prov- 
ing asymptotic normality: direct, negative roots, moments, and singularity analysis. 
In following sections we consider passing from asymptotic normality to asymptotic 
estimates for the underlying counts a(n,k), multivariate normality, and the role of 
asymptotic normality in other methods of approximate enumeration. 

The de Moivre-Laplace formulation given above in Theorem 3.2.1 is not always 
adequate for our task, and so we have ready the following generalization. 


Theorem 3.2.3 Let X;, i > 1, be a sequence of independent 0,1 random variables 
with p; denoting the probability that X; = 1, and let S,, be the nth partial sum Y"_, Xi. 
Let Ln, 0, be the mean and variance of Sy: 


n n 
Ln =) pi, Of =¥ pi(1—pi). 
i=l i=l 
Then, provided 0, — ©, 
Prob (Sn < Mn +xOn) > V(x), 


for all x. 


Theorem 3.2.1 above is the original, primordial so to speak, central limit theorem. 
One may dismiss it as a simple application of Stirling’s formula. Consider Theorem 
3.2.3, on the other hand. There are some very “combinatorial looking” expressions 
involved. Namely, one has exactly 2” distinct products 


AjA2:+:An, Ai € {pi,1— pi}. (3.2) 


Let a(n,k) be the sum of the @) products in which the first alternative ( A; = p; ) is 
exercised exactly k times. Theorem 3.2.3 asserts that the numbers a(n,k) are asymp- 
totically normal, provided the variance becomes infinite. The binomial coefficients 
have appeared as the number of products making up a(n,k), but not so prominently 
as in Equation (3.1). It is not clear that Stirling’s formula can be brought to bear in 
proving Theorem 3.2.3. 

Through the ages mathematicians have sought ways to weaken the hypotheses 
of Theorem 3.2.1 to fit the latest problem at hand. In Theorem 3.2.3 we still employ 


258 Handbook of Enumerative Combinatorics 


independent 0, | variates, but no longer are they assumed to be identically distributed. 
One may consider other improvements such as variables that assume more than two 
values, or variables that are weakly dependent, or error estimates for the convergence 
to normality. The following refinement addresses the first and last of these three. 


Theorem 3.2.4 [4, 15] Let Xj, 1 <i<n, be independent random variables with 


means |i, variances ors and absolute third central moments p; = E|X; — y;|°. With 
uw=Y;M, 0? =Y; omy denoting the mean and variance of the sum S, = Y?_, Xi, we 
have 

C Yin Pi 


|Prob (Sp <U+xo) — V(x)| < ‘ 


o3 
where C is a universal constant. 


This is the Berry-Esseen theorem. It can be used to prove asymptotic normality, 
and at the same time provide a bound on the error of the approximation. Improved 
estimates of the universal constant C have been the topic of many papers, even re- 
cently. From a purely analytical point of view, requiring the existence of the third 
moment may be regarded as an excessive assumption. But in combinatorial applica- 
tions we typically are looking at the limit of finite distributions and the availability 
of third (and higher) moments is not an issue. Thus, Theorem 3.2.4 can be a widely 
applicable tool for the combinatorial domain. It is left as an exercise for the reader to 
deduce Theorem 3.2.3 from 3.2.4. 


3.3. Method 1: direct approach 


Sometimes one can realize the enumeration problem under consideration, suitably 
normalized, as the distribution of a sum of independent 0, | variables. Then, Theorem 
3.2.3 is directly applicable. We illustrate this with three examples. 


Example 3.3.1 Let 
7 12 soe Tn 
be a permutation of the integers |n]. A left-to-right-minimum of 7 is a value 7, 
1 <i<a, satisfying 
mj = min{7;: j < i}. 
Let X;, be the number of left-to-right minima in a permutation selected uniformly 
at random. 


Theorem 3.3.2 [16, 20] With X, the number of left-to-right minima in a random 
permutation, 


Prob (x <logn +x(logn)'/?) > V(x), 


for each fixed value of x. Moreover, the same conclusion holds for the random vari- 
able Y, equal to the number of cycles in a random permutation. 
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Proof. The key to seeing that Theorem 3.2.3 is applicable is to build up the per- 
mutation 7 by placing the numbers 1,2,...,n one at a time. When 2 is placed, there 
are 2 possible positions relative to the previously placed element 1; when 3 is placed, 
there are 3 possible positions relative to the previously placed elements 1 and 2; etc. 
Letting the i possible places for the element i be equally likely means that the final 
permutation 7 will have been formed in accordance with the uniform distribution. 
In the process, we can see that the probability that 7 is a left-to-right minimum is 
1/i, and that these events are independent. The last assertion holds because whether 
or not 7 will be a left-to-right minimum is unaffected by the order of the first i — 1 
placements; it depends only on whether or not i is placed at the extreme left of the 
latter. The desired conclusion is now an immediate consequence of Theorem 3.2.3, 
with p; taken to be 1 /i, and the well-known formula 


n 


1 
y* = =logn + O(1). 
l 


n=1 


A bit more combinatorial thinking will allow us to extend the conclusion regard- 
ing X, to the random variable Y,, the number of cycles. Suppose a certain permuta- 
tion 7 is presented as an unordered set of cycles. Let the smallest element of each 
cycle serve as the cycle’s representative. Taking the cycles in the order of decreasing 
representatives, list the elements of each cycle starting with the smallest (the repre- 
sentative) and then following the cyclic order. In this manner, we go from 7 given as 
an unordered set of cycles to 7’ in one line notation, with the number of cycles of 
7 corresponding to the number of left-to-right minima of 2’. Thus, X,, and Y, have 
exactly the same distribution. | 


Example 3.3.3 This example concerns another permutation statistic, the number of 
inversions. An inversion of a permutation 1 of [n] 


Tl 2 soe Ty 


is a pair (i,j) such that i < j but m > ™j. Again taking the uniform probability 
distribution on permutations, we define random variable Z,, by letting Z,(m™) be the 
number of inversions in permutation 7. 


Theorem 3.3.4 [16, 20] With Z,(2) the number of inversions of random permutation 
T, 
Prob (Zn < n?/4-+xn*/?/6) > N(x). 


Proof. The proof is only a slight embellishment of that given for Theorem 3.3.2. 
Let the permutation 7 be built by placing the numbers 1,2,...,1 as before. When i 
is placed, it has a 1/i probability of creating j inversions with the previously placed 
numbers, where 0 < j <i. Thus, 
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where the Zo) are independent, and Zz) assumes each of the values 0,1,...,i—1 with 
probability 1 /i. Clearly, 


i i-l : n(n— | 
(Z\) = 5+ and E(Z,) = a ) 
Moreover, 
(a? Le ee ee 
(zn?) => VP =?/3-i/2+1/6. 
j=0 
whence 


Var(Z)) ~i7/12, and o? “ 


no 


Var(Z,) ~ n? /36. 


y 413 
To use Theorem 3.2.4, we need p;, the expectation of Zi) _ St | . Clearly, for some 


Cc, 
pi~ci?, and Y' pi ~ cn’ /4, 
whence 
LiPi . Sden' 9 + 0. 
On 
By Theorem 3.2.4 the proof is complete. | 


Example 3.3.5 Both the previous examples are well explained in [17], where cita- 
tions to the original papers [20] and [16] are given. Our third example is chrono- 
logically the earliest of the three, but is presented last due to its greater difficulty. 
Mark Kac gave a lecture shortly after his arrival in America about independence. 
One of his topics was divisibility by distinct primes, and he told the audience that 
there surely must be a central limit theorem awaiting demonstration. A member of 
the audience, Paul Erdés, “perked up and asked me to explain once again what the 
difficulty was. Within the next few minutes, even before the lecture was over, he inter- 
rupted to announce that he had the solution!” [24], page 90. The Erdés—Kac theorem 
marks the beginning of probabilistic number theory. 


Theorem 3.3.6 [13] Let x be a fixed real number, and let f(m) be the number of 
distinct prime divisors of m. Define 


C= #{m <n: f(m) <loglogn +x(loglogn)!/7}. 


Then, as n — ©, 

Kn 
— + W(x). 
n 
Sketch of proof: | Consider the product L = pj --- py, of all the primes less than some 
fixed bound H, and treat the integers from 1 to L as a (uniform) probability space. 
(Keep in mind throughout this proof sketch the p; are prime numbers, not probabil- 
ities.) Then the event “‘m is divisible by p;” occurs with probability 1/p;, and these 
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events are (precisely) independent. So, the probability (let us call it a) that an inte- 
ger in [1,Z] has k distinct prime divisors equals exactly the probability that the sum 
S_, = yx; of independent 0, 1 random variables X;, 1 <i<h, 
1 

Prob(X; = 1) = —, 
Pi 
be k. It is seen that the same equality holds if instead of L we consider any integral 
multiple gL of L. If instead of gL we consider a very large but arbitrary number n, 
then the equality becomes not exact, but approximate, and the error can be proven 
negligible by writing n = gL +r, with r always limited to 0 < r < L no matter how 
large n. In short, a certain density exists: 


#{m <n: fa (m) =k} 


n 


> Ak, 


where f(m) equals by definition the number of distinct prime divisors of m that are 
less than or equal to H. It is known (a weak form of Mertens’ Second Theorem) 


1 
U(Sp) = y —~loglogH, 
p<H P 


and that Var(S),) is asymptotically the same. Directly by Theorem 3.2.3 above, which 
Erdés and Kac call the “central limit theorem of the calculus of probability,” 


lim Tim *U" 52: fulm) <loglogH +x(loglogH)'/7} 


H->0n—-0o n 


N(x). (3.3) 


The hitch of getting from here to the final result, Kac tells the not necessarily 
mathematically trained readers of his autobiography, is “one of a common variety 
having to do with the difficulty of justifying the interchange of taking limits.” The 
number theoretic tool which Kac was lacking at the time of his lecture, and which 
Erdés supplied, is the theorem of Brun [6]: provided 


logn 
log Hy : 
we have 
#{m <n: mis not divisible by any prime less than H,} = (e ’+0(1)) a 
n 


The rest of the proof consists in defining H = H,, as a function of n, and in lieu of the 
iterated limit in (3.3) considering instead 


i #{m <n: fu,(m) < loglogH, +x(loglogH,)!/?} 
im ; 


n—-oo n 


Provided H,, grows sufficiently slowly with n, the limiting behavior seen in (3.3) is 
preserved; provided H,, grows sufficiently quickly with n, the two sets 


{m<n: fy,(m) < loglogH, +x(loglog H,)!/?} 
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and 
{m<n: f(m) <loglogn+x(loglogn)!/?} 


will be sufficiently close in size. Brun’s theorem given above, plus two other asymp- 
totic formulas due to Mertens [28], 


and 


(: =| e 
oer p/)  logn’ 


constitute the necessary tools. The reader may enjoy attempting the proof. Fair warn- 
ing, despite Erdés’ having found the solution in “minutes,” Kac reports (as the one 
designated to write up the proof) spending considerably more time in “assimilating 
Erdés’ proof.” | 


3.4 Method 2: negative roots 


By the fundamental theorem of algebra a monic polynomial P(x) can be factored 


P(x) =[][@+»), 


TER 


where % is the multiset of the negatives of the (complex) roots. The coefficient of x* 
in P(x) is given by the familiar formula 


kP@)= Y [In 


(Fick meee 
the sum of the products of the elements of # taken |#|—k at the time. Comparing 
this with the earlier appearing expression (3.2) leads to an interesting conclusion: if 
rj > 0, 1 <i <d, are nonnegative real numbers then 


k d XT d 
* = Prob Xi, =k], 
[x*] I] ess rO L 
where Xj, | < i < d, are independent 0, 1 random variables with 
Prob (X; = 1) : 
TO i= =, 
1 + ‘7; 


(The product of (1 +7;) on the left is needed to normalize the coefficients into prob- 
abilities.) In brief, when a polynomial P(x) has real and nonpositive roots, we have 
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a probabilistic interpretation of its coefficients. Theorem 3.2.4 is applicable. A bit of 
algebra shows that the mean and variance of )'; X; are given by the following “root- 
free” expressions 


QQ) go _ PUY(P() +P") - PO? 
He Raye Pup : 


Adapting Theorem 3.2.4 to this situation yields the following. 


Theorem 3.4.1 [21] Let 
Pa(x) = Yen kx" 
k 


be a sequence of polynomials with real nonpositive roots, and let Un,07 be the asso- 
ciated means and variances. Then, provided 0, — ~, 


1 


Oe ee ea 


asn— ov, 


Example 3.4.2 This technique was exploited by Lévy [26], and has been indepen- 
dently discovered at least one other time: Combinatorialists trace it to [21]. In 
that work, Harper studies the Stirling numbers of the second kind, S(n,k), which 
is the number of partitions of an n-set into k nonempty and pairwise disjoint subsets. 
Harper’s proof bears repeating. 


Theorem 3.4.3 [21] The polynomials P,(x) = ¥,S(n,k)x* have all real and nonpos- 
itive roots, and the Stirling numbers of the second kind are asymptotically normal. 


Proof. By a simple combinatorial argument 
S(n+1,k) =kS(n,k)+S(n,k—1). (3.4) 


(The two terms on the right count the partitions in which n + | is not, and is, re- 
spectively, in a block by itself.) Multiplying both sides by x* and summing on k, this 
translates into 

Pri(x) = x (Pi (x) +P, (x)) , (3.5) 


From here, one finds that the roots of these polynomials, (other than the common root 
x =0 which they all have), exhibit a nice interlacing property. Assume, inductively, 
that P, (x) has the roots 


Tn-1 <2 << <9 = 0. 


It is clear that P’ (x) will be alternately positive, then negative, at the points 0,r1,... . 
Then by (3.5) P,+1(x) is alternately positive, then negative, at the points r},72,... . 
We have found roots for P,+;(x) at x = 0, and in each of the n— 1 intervals 
(r1,r0); (T2,11);--+; (%n—1,7n—2)- That’s 1 + (n — 1) =n real and nonpositive roots so 
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far. The sign of P,,1(x) at r,_1 is opposite that of P’ (x), that is, (—1) x (-1)" 1! = 
(—1)”. But P,+1(x), being of degree n+ 1, has sign (—1)"*! as x —> —co. So, there is 
an (n+ 1)th root in the interval (—°°,r,_1). By induction then, each polynomial P,, (x) 
has n roots in the interval (—°,0], and these roots do indeed exhibit an interlacing 
phenomenon. 

If only we knew 0, —> , the Stirling numbers of the second kind, S(n,k), would 
be proven asymptotically normal. Let B,, the nth Bell number, equal Y,S(n,k), the 
total number of partitions of [n]. Using (3.4), as found by Harper, 


Un = Bnii/Bn —_ 1, oe = Bni2/Bn a (Bn+1/Bn)* = (3.6) 


Some careful asymptotic estimates for B,, complete Harper’s proof. | 


In colorful language Harper says that traditionally proofs of asymptotic normal- 
ity proceed by “torturing the characteristic function until it converges to et / >” (See 
Section 3.5). But in the present proof, one needs a “hat from which to pull a rabbit,” 
and that hat is the central limit theorem. With no intention to diminish the magical 
aura of the proof, might not one nevertheless seek a definite combinatorial meaning 
for the probabilities 1/(1-++r;)? That is, perhaps we can define some independent ran- 
dom variables Y; with Y’; Y; having the desired distribution, and apply Theorem 3.2.3 
directly. Think back to the first example of the direct method: the number of cycles 
in a random permutation. There we let each cycle be represented by its smallest ele- 
ment, and produced a listing of numbers whose left-to-right minima corresponded to 
the cycles. A similar process suggests itself for partitions of [n]. Let the partition 2 
be given as an unordered collection of subsets, usually referred to as blocks. Repre- 
sent each block by its smallest element, and in this way the blocks may be numbered 
1,2,...,k, k being the number of blocks. Now define a function fz : [n] — [k] by let- 
ting all members of the ith block be mapped by fz to i. Make a list of the values of 
this function 


fr(1) fr(2) +++ fr(n). 


It will be seen that f; is a restricted growth function, meaning 


fr(i) < 14+max{ fr(j) : j <i}, 


and that the number of blocks of partition 7 is exactly the number of left-to-right 
maxima of the associated restricted growth function f;. Let the set of all restricted 
growth functions on |n] have the uniform probability, and define the 0,1 random 
variables Y; to be the indicators of the event “a left-to-right maximum for f occurs at 
position i.” We have just seen that 


oe (Ex =8) [ su 


so the sum )’;Y; has the desired distribution. Unfortunately, the Y; are not indepen- 
dent, and the theorems available to us do not apply. (The one-line notation for set 
partitions, widely used now, appears first in [32, page 96].) 
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Example 3.4.4 We close this section with an example taken from the theory of 
matchings in graphs. Let G = (V,E) be a simple graph; a subset M C E is called 
a matching in G provided no two distinct edges e,e' € M have a vertex in common. 
Let (G) be the set of all matchings in graph G, and define the matching polynomial 
P(x) by 


Pox) yl, 
Me.m(G) 


Thus, |x*|Pg(x) equals the number of matchings in G of size k. In addition to its graph 
theoretical origins, this concept has been a long-time interest in statistical physics 
where the terminology is that of monomer-dimer coverings. The remarkable fact 
that Pg(x) always has all its roots real and negative is due to this latter community. 
See the seminal paper of Heilmann and Lieb [23]. In that work Heilmann and Lieb 
prove a more general assertion using different polynomials, namely 


Oo) = YY (-1) Mw (ayer 2M, 
Mc. (G) 


where whenever used as a subscript of the symbol “Q” G is a complete graph on n 
vertices whose edges carry positive weights, and W(M) is the product of the weights 
of the edges comprising M. For a vertex i in the edge-weighted complete graph G, 
partition (G) into those M which, respectively, do not and do cover vertex i; one 
is led to the recursion 


QG(x) =xQc-i(x) — Y wijQe-i-j(x). 


je[n)-i 


(wij is the weight on edge ij.) From here an argument resembling that given ear- 
lier for S(n,k) establishes by induction that the roots of Qg-; are real, distinct, and 
interlace those of Qg. By taking a limit in which all edge weights are I and 0, one 
finds that x" Pg(—1/x*) (G now arbitrary) has all real, albeit not necessarily distinct, 
roots. Whence the matching polynomial Pg(x) has all roots real and nonpositive. 


Godsil [18] took up this topic and, building on Harper’s work, found the follow- 
ing version of Theorem 3.4.1 for the matching polynomials. 


Theorem 3.4.5 [18] Let G, be a sequence of graphs, X, be the size of a matching 
M drawn uniformly at random from (Gn), Un = E(Xn), and 62 = Var(X,). Then, 
provided 0? + ~, 


Prob(Xn < Un +xOn) > V(x). 


Two applications are noted: G, a sequence of regular graphs of degree r, with 
the number of vertices becoming infinite; and G, equal to the complete graph, Ky. 
Later, Kahn [25] continued the study of this topic, and gave, among other things, a 
formulation of the condition o,, — °° in purely graph theoretical notions. 
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Theorem 3.4.6 [25] Let G,, be a sequence of graphs, X,, be the size of a matching M 
drawn uniformly at random from (Gn), Un = E(X,), and Vy, the size of the largest 
matching in M(G,). Then 


Prob (Xn <n +XOn) > V(x) 


if and only if 
Vn—-Un 7 ©. 


3.5 Method 3: moments 


For a random variable X, the associated moment generating function is defined by 


My(t) & Ee™, 


No question of convergence arises when T is pure imaginary, but t = 0 could be the 
only real value for which Mx(t) is defined. For X distributed as standard normal, 
Mx(t) is the entire function e* /2, When Mx(t) is analytic near tT = 0, it has the 
power series expansion 

srk af 

kl 


hence the nomenclature “moment generating function.” The following theorem finds 
frequent combinatorial application. 


Theorem 3.5.1 [11] Suppose X,, is a sequence of random variables with distribution 
functions F,(x) such that 


2 
7 etXn +e /2 


for all T in a nonempty real interval (—a, +a). Then, for all x, 


Fi, (x) > V(x). 


Similar limit theorems are known for the characteristic function Ee’ (Lévy’s 
Continuity Theorem, [5], Theorem 26.3) and for moments (the Fréchet-Shohat The- 
orem, [5], Theorem 30.2). The theorem for moments states that if each member of se- 
quence X,, has moments of all orders, and for each k the kth moments of X;,, converge 
to the kth moment of the standard normal, then the X;,, are asymptotically normal with 
mean 0 and variance 1. 


Example 3.5.2 Let g(n,k) be the number of partitions of n into exactly k distinct 
parts. For example, q(8,3) = 2, the two partitions being 5 +2+1 and4+3+1. The 
following theorem is stated without proof in [14], excluding the explicit formula for 
the variance. 
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Theorem 3.5.3 [14] Define 


whe 2V3In(2) 1/2 
1 
2 
6 2 ye 1 (eee) ni? 
1 1 


Then the numbers q(n,k) are asymptotically normal with mean [ln and variance 07. 


Proof. Define the polynomials Q,(y) = Y,.q(n,k)y*, and the generating function 
f(uy) = Yx"Qay) = [] +2”). 
n=0 
By Theorem E it suffices to show 
et /2, (3.7) 
Using contour integration around a circle of radius e~‘, t > 0, we have 
—nt © i a —t+id Tt) ,—nid 
e"Q,(e") = =| fle eye" dé. 
20 J—x 

Throughout the remainder of the proof, asymptotic formulas hold uniformly for 
n—> co and t,t > 0 with t = (cy/n)!/? and t = O(t'/?). (The constants co,c1,¢2 
are respectively 1/2,In2,27/12.) 

Using 

Fett ere wf (e,e7) exp (cy tt 710 — cot 707), [O, <7, (3.8) 


we find 


417/5 ; ; 7 ce 
/ fle ete" do ~ | F(etie*) 8/2 exp( —S1. 4-1). 
_ 71/5 fom) 4c2 


Using 
Ife e")| < fleteNe ll", c>0,175<|ol<n, (3.9) 


we find that the integral over P< || < z is negligible by comparison. This gives 
an asymptotic for e~””Q,,(e"), which we use twice to find 


Q,(e") flee") ie ee 
On) ~ fle] exp (— 57 ) 


Finally, using 
fle“,e*) 


’ -1 , 0,9 rs) 
fle] exp (cit +—=T : (3.10) 
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we reach the desired conclusion (3.7). The three key formulas (3.8), (3.9), and (3.10) 
can be proven either by Euler-Maclaurin summation, or by the Cahier-Mellin for- 
mula; see [8], for a similar calculation. | 


The value for 67 appears in [31]. In the latter work, Szekeres proves a formula 
equivalent to 
—x?/2 
G,(2m)'/2 Qn(1), k=bMn+xOn, 
for k— Un, = o(n'/3), That is, he proves a local limit theorem; see Section 3.7. 
In [14] Erdés and Lehner concentrate on p(n,k), the analogous quantity for un- 
restricted partitions. These numbers are not asymptotically normal, and it would 
be interesting to follow the methodology of Theorem 6 to see where the differ- 
ence arises. Erdés and Lehner’s derivation of the limiting distribution of p(n,k) is 
a brilliant application of inclusion/exclusion—reminiscent of Brun’s sieve in Exam- 
ple 3. Another theorem about partitions, similar in conclusion to Theorem 6, has 
been given by Goh and Schmutz [19]: The number of distinct parts in a random 
unrestricted partition is asymptotically normal with mean (./6/ )n!/ > and variance 


(v6/(2z) — y54/n°) nil, 


Example 3.5.4 Let g;, i > 1, be a sequence of nonnegative integers, and define a 
doubly-indexed array a(n,k) by the equation 


def 1 n 
a(n,k) = i an ON a (3.11) 


e 
q(n,k) ~ 


Combinatorially, a(n,k) is the number of ways to partition |n] into k blocks, and 
to build a g-object on each block, g; being the number of g-objects on an i-set. If, 
for instance, we let g; equal I for all i, then the above becomes a familiar formula 
for S(n,k), the number of partitions of |n| into k blocks. As another illustration, if 
gi =(i—1)!, the number of full cycles on an i-set, then the above a(n,k) is the number 
of permutations of |n| having k cycles. Letting g(u) be the exponential generating 
function of the sequence gj, 


su) = Dare, 


i>l 


the generating function equivalent of (3.11) is 


ES (Senan'] = exp(ve(u)). 


The polynomials P,(y) = Y,a(n,k)y* are polynomials of binomial type [30]. 
Theorem 3.5.5 [7] Let 


d d 
g(u) = y? gju!/j! = Yi ju! 
j=1 j=l 
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be a real polynomial with nonnegative coefficients, at least two of which are nonzero, 
and such that 

gcd{j:cj AO} = 1. 
Let the doubly-indexed array a(n,k) be defined by (3.11). Then the numbers a(n,k) 
are asymptotically normal. 


Sketch of proof. As in the prior example, relying on Theorem 3.5.1, it suffices to 
define Ln, 67 that satisfy 


T/On 
-tHn |, Pe) a2 (3.12) 


P,(1) 


Again, we begin with contour integration around a circle to extract P,(y)/n! as the 
coefficient of u” in exp(yg(u)), and, as before, the integral is dominated by a small 
arc near the positive real axis, provided the radius p is chosen by the saddlepoint 
equation 

ps (p) = n/y. 


Let r(X) denote the inverse function of Xg’(X), so the latter radius is p = r(n/y). 
The estimate resulting from the contour integration is 


nPaly)  explyg(r(n/y)) 
r(n/y) aes (mB)? n>o, yl, 
with B = d’cgr(n)¢ (recall d is the degree of g(u)). In Example 3.5.2, the next step 
was to form the quotient, but here a nice feature of Example 3.5.2 is missing: Even 
for y + 1 we must keep up with both r(m) and r(n/y), and we cannot use the same 
radius for both P,(1) and P,(y). However, the common terms, B and n!, do drop out 
of the quotient leaving 


Paty) expvatr(n/y)) (7) \" 
Bi) ~ “epight) (apy) | PIM IAH 


The next step is to recognize the right side of this formula as 


exp (fe(rin/B)) 4B). 


Looking back at (3.12), we need to define Ly, o so that 


et/On 


2 
+f alrtn/pyap > 5. 


On 


The expansion of g(r(n/B)) about B = | begins 


2 


s(r(n/B)) = g(r(m))—"(B-1) + 
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and so we are led to define 

Hn = g(r(n)) 

OG, = a(r(n))——. 


The rest of the proof involves looking carefully at the higher-order derivatives of 
the function B +> g(r(n/B). The number of derivatives that come into play, call it K, 
must satisfy KJ/2 > d, where J is the second highest index (after d) such that cy 40. 
Details can be found in [7]. | | 


Applications of Theorem 3.5.5 include partitions into blocks of bounded size, 
permutations with cycles of bounded size, and permutations whose order divides a 
givend > 1. 


3.6 Method 4: singularity analysis 


The next theorem furnishes an example of a simply stated criterion on the bivariate 
generating function f(x,y), which implies asymptotic normality for the coefficients 
a(n,k). In this case, the criterion is that the function have only one singularity on the 
circle of convergence, and that it be well behaved. 


Theorem 3.6.1 [3] Let f(x,y) = Lx, a(n,k)x"y*, with a(n,k) > 0. Suppose there 
exist 


(i) a function A(s) continuous and nonzero near 0, 
(ii) a function r(s) with bounded third derivative near 0, 
(iii) a nonnegative integer m, and 


(iv) positive numbers € and 6 such that 


(1-4) "ree - 405 
x|<r(0)+6. 


is analytic and bounded for |s| < €, 


Put « = —r'(0)/r(0) and 0? = nw? — r"(0)/r(0). If o? £0, then the numbers 


a(n,k) are asymptotically normal with mean Un and variance O-n. 


Sketch of proof. Let 
flee) =Lonls)z". 
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If a power series )\a,z” has finite radius of convergence r, and f(z) —A/(1 —z/r)” 
is analytic in a larger circle, then a, ~ An’~!r~"/(m—1)!. By this sort of analysis it 
is shown 


On(s) ~A(s)n™r(s) "/m!, 


uniformly in s. The characteristic function is 


enittin/o, OCi/On) (= a) 


9(1) r(0) 
and by expanding 
r(s) r'(0) r(0) (0) \? 
(0) oo ( See (45 (xa) Jems) 
the proof is completed. | 


Example 3.6.2 Say that a permutation 1 of |n] has a rise at position i, 1 <i <n, if 
Tj < 141. Forn > 1 we follow the (fairly) standard practice of declaring there also 
to be a rise at position i = 0. Let A(n,k), 1 << k <n, be the number of permutations 
of |n| having k rises. These are called the Eulerian numbers. 


Theorem 3.6.3 [3, 12] The Eulerian numbers A(n,k) are asymptotically normal 
with mean My = n/2 and variance 02 = n/12. 


Proof. | We start with the two-variable exponential generating function as given by 
David and Barton [12] 


def an 1l-y 
f(y) = Leon =o) (3.13) 


By solving for the vanishing of the denominator, we find that the radius of conver- 
gence for f(x,e*) is 


r(s) = Bu =1-—5/2+s*/12+-.. 


e—1 
Theorem 3.6.1 is applicable, and r(0),7’(0),r"(0) equal 1,—1/2,1/6 respectively. 
Hence, uw = 1/2 and o? = (1/2)? — 1/6 = 1/12. | 


Remarks. The asymptotic normality of the Eulerian numbers can be, and has been, 
obtained in other ways. Let E,,(y) = ¥,A(n,k)y* be the associated polynomials. With 
the convention that Eg = 1, the following recursion can be deduced by combinatorial 
reasoning: 


A(n,k) = kA(n—1,k)+(n—k+1)A(n—1,k-1), n>1. 


From here we find 


En(y) = yA—-y)E,1(”) + nyEn1, n>1. (3.14) 
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By taking derivatives with respect to y, evaluating at y = 1, and solving the resulting 
recursions, one can obtain the two exact formulas 


= n+1 

In = 5) 
n+1 

oe = ; 


for the mean and variance. Using the technique illustrated in Section 3.4 one can 
prove from (3.14) that the polynomials E,,(y) have interlacing real negative roots. 
Hence, by Theorem 3.4.1, the Eulerian numbers A(n,k) are asymptotically normal. 
This is the approach taken in [9]. 

Another approach is that taken by David and Barton [12]. Working with the two- 
variable exponential generating function (3.13), they prove asymptotic normality us- 
ing a variant of the method of moments, the method of cumulants. 


3.7. Local limit theorems 


Suppose a doubly-indexed array a(n,k) is asymptotically normal with mean 1, and 
variance 67. This may tell us nothing about an individual number a(n,k). For in- 
stance, in Example 3, concerning the number, f(m), of prime divisors of a random 
integer m, we dare not hope for an asymptotic formula for the number of prime divi- 
sors of m. Indeed, we suspect that integers with one prime divisor may very well be 
adjacent to an integer with an extraordinary number of prime divisors. Just knowing 
the frequency with which f(m) assumes various values, the very content of asymp- 
totic normality, is a surprising and pleasing amount of information for such a noisy 
function. 

On the other hand, suppose for a given n the numbers a(n,k) do not vary chaoti- 
cally like the divisor counting function; suppose instead they vary smoothly. Let J, 
be the discrete set of real t in the interval (—°°,x] with the property that U, +10, is 
an integer. The t € Y, are spaced at a distance 1/0, and if 6, + ©, 


1 y et 
Gn te, V 2n 


But by asymptotic normality 


> N(x). 


a(n, Mn a tOn) 
tEeD. LY. a(n,k) 


Here we have two smoothly varying sequences of numbers whose partial sums are 
asymptotic; optimistically, one hopes 
a(n,k) et /2 | 
ya(n,k) On V2 = Hn) / On’ 


> N(x). 
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This prompts the next definition. 


Definition 3.7.1 We say the doubly indexed sequence a(n,k) satisfies a local limit 
theorem on the set S of real numbers provided 


—x?/2 
eee Ona(n, | Mn +XOn |) e ay 
xeS Ye a(n,k) V20 


Theorem 3.7.2 [3] Suppose that a(n,k) are asymptotically normal, and 0? — ~. If 
for each n the sequence a(n,k) is unimodal in k, then a(n,k) satisfy a local limit 
theorem on the set {x : |x| > €}, any € > 0. If for each n the sequence a(n,k) is log 
concave in k, then a(n,k) satisfy a local limit theorem on the set R. 


Unimodality, 
a(n,1) <+++<a(n,K) >a(n,K+1)>--, 


and log concavity, 
a(n,k)? > a(n,k+1)a(n,k—1), 


are features that arise often in combinatorics. Both of these constitute “smoothness” 
adequately enough to pass from a central to local limit theorem, as stated in Theorem 
3.7.2. 


Example 3.7.3 If P,(y) =Y,.a(n,k)y* has all its roots real and nonpositive, then its 
coefficients are log concave. Thus, from Examples 3.4.2 and 3.6.2, we derive genuine 
asymptotic formulas for Stirling numbers of the second kind S(n,k) and Eulerian 
numbers A(n,k) when (k — Mn) /On is bounded. 


Theorem 3.7.4 [21, 29] Let Un =n/Inn and o? = n/(Inn)*. Then, uniformly for 
k— Un = O(n), 


S(n,k) en ¥n/2 
B,~ GalOm ier t= Re) /On. 


Proof. To obtain the formulas for LU, oO, we use Equation (3.6) and an asymptotic 
estimate of the Bell number B,,. Then, Theorem 3.7.2 is applicable. | 


Theorem 3.7.5 [9] Let ly, =n/2 and 02 =n/12. Then, uniformly for k — Un = 
O(Gn), : 

A(n,k) ewtn/? = 

ml! Gada? 2 =~ Hn)/On- 


Proof. Derivation of the formulas for [,,67 has been indicated earlier. Then, The- 
orem 3.7.2 is applicable. | 
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Remark 3.7.6 Harper [21] states Theorem 9.1 as a corollary to his central limit theo- 
rem for S(n,k), giving a quite succinct and informal justification. The first asymptotic 
formula for S(n,k) covering all k was provided earlier by Moser and Wyman [29]. In 
Moser and Wyman’s formula the connection to the central and local limit theorems 
is not at all apparent unless one specializes k to be near Ll, and does the necessary 
algebra. In their work [9] Carlitz et al. acknowledge their debt to Harper, and utilize 
the Berry-Esseen theorem in deriving an asymptotic formula for A(n,k) with explicit 
error bound. Their proof, which Riordan complements in his math review, can be 
read as a careful fleshing out of Harper’s succinct proof. Theorem G, due to Bender, 
can be read as an adaptation of the local limit theorem in [9] that does not use the 
Berry-Esseen theorem and that holds in perfect generality. 


3.8 Multivariate asymptotic normality 


With the intent of providing the reader with the basic and most widely used princi- 
ples of our topic, the discussion thus far has been totally one-dimensional. Of course, 
there is such a thing as a d-dimensional Gaussian distribution, and multivariate dis- 
tributions are ubiquitous in combinatorial settings. 


Definition 3.8.1 Let k,i be d-dimensional vectors of integers, and a(n,k) an array 


of nonnegative numbers. We say that a(n,k) is asymptotically normal with mean 
m, and covariance matrix B,, if 


—> 0. 


a(n,i) 1 i: ( ss : 
sup exp | —=xB, x’ | dx 
u ove La(n, k) (27)4/2 Bn | ue x<u TAG 


For a d-dimensional normal Z with mean 0 and covariance matrix B (the proba- 
bility density in the second operand above), the moment generating function is found 


to be 
y (e*) = exp (;780") : 


As in the one-dimensional case, one may prove convergence to normality by the 
method of moments. For instance, let each of €,,64,...,6, be a d-dimensional real 
vector, and consider the random variable X = (x),...,xq) given by 


X = &; with probability c;, 1<i<f, Yoj=l. 


L 


Form the ¢ x d matrix M using (c;)!/2&, as the ith row: 


(M)ij = (ci)"? (Ej; 
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then, the (j,k) entry of the product M7 M is E(x;x;). With m the vector of means, we 
find 


1 
Ee™* = exp (ms + 57 (M'M —N) 4015 5). 
J 


where (N) jx = (Ex;)(Ex,). Now B = M'M —N is the covariance matrix for X, and 
it follows that the sums )""_, X; (with the X; independent and each distributed like X) 
are asymptotically normal with mean nm and covariance matrix nB. So, we have a 
d-dimensional analog of Theorem 3.2.3 courtesy of the method of moments, and we 
might reasonably anticipate products g(x)” to be likely sources of normality. 

In [1] it is shown that if the coefficients @,,(x) in the ordinary generating function 


Yia(n, k)z"xk — oy On(x)z” 


behave asymptotically like a product a@,h(x)g(x)", then under suitable general con- 
ditions the a(n,k) will satisfy a central limit theorem. Generalizing Theorem 3.6.1, 
Bender and Richmond prove in [1] that this principle applies to generating functions 


f (x, z) for which 
eee A(s) 
(75) 19 - hw 


is analytic and bounded for || s ||< €, |z| < |r(0)|+6, with A(s),r(s) satisfying condi- 
tions much as in Theorem F. As an example, they apply this theorem to the generating 
function 

ky ko cl = XS +.x2 (x1 + 1)c 


f (x1,%2,2) Yia(n, 1; 2) x1 X54 n! 1— (x +x1)C ’ 


in which a(n,k,,k2) is the number of permutations of [n] having k; rises in odd 
positions and kp falls in even positions [10]. Here, 


C =cosh(Bz)—1)/B?, S=sinh(Bz)/B, B = \/G2—N—D. 


They deduce that a(n,k,,k2) are asymptotically normal with mean equal to nm and 
covariance matrix equal to nB where 
= — | 1/8 1/12 
m = (1/4,1/4), B= (Yio 1/8 i 
As another example, consider a(n,k,,k2) equal to the number of matchings in 


the product graph [n] x [2] consisting of k; horizontal and kz vertical edges. Then the 
generating function is given by 


l+x, 1 1 1 
x2 0 x O 
3 a(n, kiko) 2"x4 x? = Ne 
nkpko n>0 iS 2 0 0 
2 0 0 0 


276 Handbook of Enumerative Combinatorics 


(indicating the extraction of element (1,1) from the matrix) [2]. For a great variety 
of such matrix recursions, the authors of [2] show that both central and local limit 
theorems hold. The mean and variance are again proportional to n, but the details of 
how to calculate these are not easy and the interested reader is asked to consult the 
original paper. 


3.9 Normality in service to approximate enumeration 


Let g(u) =y gjul be a power series with real, nonnegative coefficients having radius 
of convergence R, 0 < R < ~. For each r < R we may consider the random variable 
X, which assumes the value j with probability g;r//g(r). Define 


a(r) = rel(r)/e(r), br) S ra'(r); 
it will be seen that these are the mean and variance of X,;. Assuming X; to be asymp- 
totically normal gives 
1 ; 
— yo gir a N(x), rR. (3.15) 
s(r) j<a(r)+xb(r)!/2 


Let the sequence r,, be defined by the equation 
a(ry) =n. 


If r, — R, and if passage from central to local limit theorem is permitted, we have 
(after isolating g, on one side) 


(Tn) 


7 (2mb(ra)) V2’ (3.16) 


&n™~ 


n being a(rn) +0 x b(rn)!/? and -V (0) being (27)~!/?. In [22] Hayman has axioma- 
tized, so to speak, a large class of power series g(u) for which the conclusions (3.15) 
and (3.16) hold true. Hayman calls these functions g(u) admissible. We present the 
exact definition in a moment, but first the prototypical example: g(u) = e“. Then, 
a(r) =r, b(r) =r, rn =n, and from (3.16) 


1 e” 
n! nn” (2an)!/2’ 
Stirling’s formula (upside down). Hence the title of [22]. Now for the definition. 


Observe that the characteristic function of X, is g(re’®)/g(r); by Section 3.5 a 
necessary and sufficient condition for X, to be asymptotically normal is 


e 18alr)/0(0)" g (pei /B(r)"!?) fa(p) =% ge 
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Definition 3.9.1 A power series g(z) convergent for |z| < R, with 0 < R < »%, is ad- 
missible provided we have a function 5(r), 0 < d(r) < a, defined for Ry <r<R 
such that 

g(re!®) ~ g(r) exp(i@a(r) — (1/2)0*b(r)), asr > R, 


uniformly for |O| < 8(r), while uniformly for 5(r) < |0| < x, 


io) _ 0(8(7)) 
j= Bry? asr— R. 
It is also required that 
b(r) > +00, asr> R. 


Deriving (3.15) and (3.16) for an admissible function is a simple application of 
the circle method. The value of Hayman’s paper lies in its characterization of a large 
class of power series to which the circle method is applicable. Numerous properties 
of admissible functions are proven, as well as closure of the class under a number of 
operations. The paper provides a template of how to use the circle method effectively. 
For example, the proofs in Section 3.5 are found by making simple modifications of 
this template. The later and deeper sections of the paper have not been explored by 
combinatorialists whose immediate intent is applications to enumeration. 

In later years McKay, jointly with a variety of co-authors, has developed a new 
method of asymptotic enumeration that follows this same general pattern. The semi- 
nal example is the generating function 


f(x) = [J] Ct+aix;), 


1<i<j<n 


treated by McKay and Wormald in [27]. The coefficient 


[aft --x@] f(x) 


is the number of simple, labeled graphs on n vertices having degree sequence 
d,,...,d,. For certain degree sequences the coefficient can be estimated by the multi- 
dimensional circle method. For the multi-dimensional integral a set of radii 7r1,...,7n 
is needed, and these are found by solving a system of saddlepoint equations. The 
main contribution to the integral might arise from x; = rje% with |9;| all small, but in 
some instances a host of small contributing regions is identified. Around the points 
that do contribute, the shape of the integrand is found to be asymptotically normal, 
now a multivariate normal, and the estimation of the primary integral involves diag- 
onalizing the Hermitian covariance matrix. Complications arise due to the fact that 
the number of variables is growing along with n. This means that the Taylor series 
used in the primary integrals cannot stop with a linear and quadratic, but must go 
at least, typically, to quintic. Estimating the integral of these quintic terms is a new 
challenge. The necessary argument that the contribution to the integral from outside 
the primary region is negligible becomes quite complicated and combinatorial in na- 
ture. Although similar issues recur in each new application of this method, no useful 
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general systematization has yet been proposed. The intermediate algebra and anal- 
ysis can become too large to handle without computer-based methods, yet the final 
result is typically a succinct and elegant asymptotic formula. These formulas strik- 
ingly agree with formulas found for other ranges of the parameters and proven by 
totally different methods. 
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4.1 Introduction 


Trees are a fundamental object in graph theory and combinatorics as well as a basic 
object for data structures and algorithms in computer science. This is certainly the 
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reason why (several types of) trees and tree-like structures are of constant interest in 
research. 

In this chapter we will survey and explain several methods for the enumeration 
of trees. The main focus is, however, the method of generating functions that is a 
very appropriate and powerful tool for tree enumeration. This is mainly due to the 
recursive structure of trees so that combinatorial decompositions can be easily trans- 
lated in corresponding relations for generating functions. We, thus, decided to use 
the generating functions terminology throughout the chapter. 

After an introductory section we concentrate on the enumeration problem of un- 
labeled trees and that of labeled trees separately. A concluding section is devoted to 
three selected topics that are related to trees and tree enumeration. 

The literature on trees is vast, in particular on algorithmic questions and on prob- 
abilistic properties of trees. There are also several books that are specialized on tree 
enumeration. Probably, the first book is due to Moon [21] from 1970 that specializes 
on labeled trees. A few years later the classical book by Harary and Palmer [18] on 
Graphical Enumeration was published. Trees are an an important part of that book, 
but the book goes much beyond them, and generating functions are widely used. An- 
other milestone is the book by Bergeron, Labele and Leroux [3] on Combinatorial 
Species and Tree-Like Structures (that was first published in French in 1994 and then 
in an English version in 1998). It develops a theory for the enumeration of unlabeled 
tree-like structures by the use of Pélya’s theory of counting. Finally the author wants 
to mention two more recent books. The book by Flajolet and Sedgewick [13] on 
Analytic Combinatorics systematically uses the relation between combinatorial con- 
structions and generating functions (we adopt this philosophy in this chapter) and 
trees and tree-like structures are discussed there in various aspects. Another focus in 
[13] is the asymptotic analysis of combinatorial objects based on generating func- 
tions. Finally we also refer to the book [11] on Random Trees by the author, where 
the approach of [13] is extended to a systematic analysis on the stochastic behavior 
of tree parameters under various random tree models. Again generating functions (in 
particular in several variables) are used as a basic tool. Some parts of the present 
chapter follow the presentation of [11]. 

In the following text there are several references to the aforementioned books 
(and, of course, also to other sources) since the limited space of this chapter can 
only provide an introduction to tree enumeration. The choice of the presented results 
reflects the authors’s taste, too. So the author encourages the interested reader to 
consult in particular these references in order to obtain more detailed information on 
more refined enumeration problems on trees. 
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4.2 Basic notions 


Trees are finite simple connected graphs without cycles. If we just require that a finite 
simple graph has no cycles then it is called forest. The connected components of a 
forest are then trees. 

The degree d(v) of a node v in a tree is the number of nodes that are adjacent to 
v or the number of neighbors of v. 

Nodes of degree < 1 are usually called leaves or external nodes and the remain- 
ing ones are internal nodes. If a leaf has degree 0 then the tree consists just of this 
vertex and has no edges. In all other cases leaves v are characterized by d(v) = 1. It 
is easy to see that leaves always exist. 


Lemma 4.2.1 Every tree with at least two vertices has at least two leaves. 


Proof. We fix an arbitrary vertex vg of the tree T to start a path at v that uses an edge 
of T at most once. Since T has no cycles no vertex will be visited twice. Furthermore 
the path can be extended as long as the degree of present vertex is > 2. However, 
since a tree is a finite graph such a path cannot be infinite. Thus it has to terminate at 
a vertex with degree 1, which is a leaf. If the starting point vo was already a leaf then 
we are done. In the other case we just start a second path at vo and will find a second 
leaf. a 


Lemma 4.2.1 can be used to deconstruct a tree by removing edges. Since a leaf 
has degree | there is precisely one edge attached to it. So if you remove the leaf 
and the corresponding edge the remaining graph is still a connected and cyle-free 
graph. In this way it follows by induction that we construct any tree by starting with 
a single vertex and by adding step by step an edge (together with a new leaf). This 
also implies the following property of trees. 


Lemma 4.2.2 Let OQ denote the number of vertices of a tree T and O the number 
of edges of T. Then we have 
a =a-—1. 


Conversely if a simple connected graph has the property that the number of edges is 
1 less than the number of vertices then the graph is a tree. 


Proof. — As already mentioned the relation @ = Q@ — 1 follows by induction. Next 
we observe that every connected graph G has a spanning tree 7, that is, a maximal 
subgraph G that is a tree. It is easy to see that T contains all vertices of G. Since T 
is a tree the number of edges of T equals the number of vertices minus |. Hence, T 
contains all edges of G, too. Thus T and G coincide. a 


Another characterization of a tree is the property that for every pair of vertices 
there exists a unique path that connects these two vertices. 
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Figure 4.1 
Tree and rooted tree. 


Rooted versus unrooted trees. If we mark a specific node r in a tree T, which 
we denote the root of T, we call the resulting tree a rooted tree. See Figure 4.1 for 
an illustration. A root induces also a kind of hierarchy in terms of generations or 
levels. The root is the zeroeth generation. The neighbors of the root constitute the 
first generation, and in general the nodes at distance k from the root form the kth 
generation (or level). If a node of level k has neighbors of level k + 1 then these 
neighbors are also called successors. The number of successors of a node v is also 
called the out-degree d* (v). For all nodes v different from the root we have d(v) = 
dt(v)+1. 

Furthermore, if v is a node in a rooted tree T then v may be considered as the root 
of a subtree 7, of T that consists of all iterated successors of v. This means that rooted 
trees can be constructed in a recursive way. Due to that property counting problems 
on rooted trees are usually easier than on unrooted trees. 

Rooted trees also have various applications in computer science. They naturally 
appear as data structures, e.g. the recursive structure of folders in any computer is just 
a rooted tree. Furthermore, fundamental algorithms such as Quicksort or the Lempel- 
Ziv data compression algorithm are closely related to rooted trees, namely to binary 
and digital search trees which are also used to store (and search for) data. Rooted 
trees even occur in information theory. For example, prefix free codes on an alphabet 
of order m are encoded as the set of leaves in m-ary trees. 


Plane versus non-plane trees. Trees are planar graphs since they can be embedded 
into the plane without crossings. Nevertheless, a tree may have different embeddings 
(compare with Figure 4.2). This makes a difference in counting problems. When we 
say that we are counting planar trees we mean that we are counting all possible 
different embeddings into the plane. 

In the context of rooted trees it is common to use the notion plane tree or ordered 
tree when successors of the root and recursively the successors of each node are 
equipped with a left-to-right-order. Alternatively one can give the successors a rank 
so that one can speak of the jth successor (j > 1). Of course, this induces a natural 
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Figure 4.2 
Two different embeddings of a tree. 


Figure 4.3 
Plane rooted tree. 


embedding into the half-plane (compare with Figure 4.3). Note that this notion is 
different from considering all embeddings into the plane, since it is not allowed to 
rotate the subtrees of the root cyclically around the root. 


Labeled versus unlabeled trees. We also distinguish between labeled trees, where 
the nodes are labeled by different numbers, and unlabeled trees, where the nodes 
are indistinguishable. This is particularly important for the counting problem. For 
example, there is only one unlabeled tree with three nodes whereas there are three 
different labeled trees of size 3 with labels 1,2,3. 

There is much latitude in choosing labels on trees. The simplest model is to as- 
sume that the nodes of a tree of size n are labeled by the numbers 1,2,...,”, but 
there are many other ways to do so. For so-called embedded trees one only assumes 
that the labels of adjacent vertices differ (at most) by 1. Another possibility is to put 
labels consistently with the structure of the tree. For example, recursive trees have 
the property that the root is labeled by 1 and the labels on all paths away from the 
root are strictly increasing. 
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4.3 Generating functions 


The concept of generating functions is a very useful tool to count combinatorial ob- 
jects that can be characterized with the help of proper combinatorial constructions. 
However, generating functions are not only useful in counting but they can be used, 
too, to obtain asymptotics by evaluating the Cauchy integral (that computes the coef- 
ficient of a power series) asymptotically. Finally, but this will be not covered in this 
chapter, they can be used to encode the distribution of random variables that are re- 
lated to counting problems and, hence, asymptotic methods can be applied to obtain 
probabilistic limit theorems like central limit theorems. 

In this chapter we first survey how certain combinatorial constructions have their 
counterparts in relations for generating functions which can then be used to count 
the original combinatorial objects and present also three special topics (Lagrange 
inversion, the Dissymmetry Theorem, and Asymptotics) that are related to generating 
functions and have several applications to tree enumeration. 


4.3.1 Generating functions and combinatorial constructions 


We present very briefly a systematic approach of combinatorial constructions and 
generating functions. The presentation is inspired by the work of Flajolet and his 
co-authors (see in particular the monograph [13] where this concept is described in 
much more detail.) * 


Definition 4.3.1 The ordinary generating function (ogf) of a sequence (An)n>0 (of 
complex numbers) is the formal power series 


A(x) = ) Apx”. 


n>0 


Similarly the exponential generating function (egf) of the sequence (An)n>o is given 
by 


We use the notation 


to read off the coefficient of x” in a generating function. 

It is clear that certain algebraic operations on the sequence A,, have their counter- 
part on the level of generating functions. The two tables in Figure 4.4 collect some 
of them. 

*More detailed descriptions can be found in Chapter 1 on Algebraic and Geometric Methods in Enumer- 


ative Combinatorics and in Chapter 2 on Analytic Methods. In order to keep the chapter self-contained we 
keep this short overview of notions that we will need for the enumeration problems on trees. 
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sequence ogf 
sum Cr =An+Bn C(x) = A(x) + B(x) 
n 
product Cn = ¥ AgBn—k C(x) = A(x)B(x) 
k=0 
n 
partial sums Cr= ¥ Ag C(x) = ~KA(a) 
k=0 
marking Ch = nAn C(x) = xA’(x) 
scaling Cy = c"An C(x) = A(cx) 
sequence egf 
sum Cr =An+Bn C(x) = A(x) + B(x) 
product Cr= LZ ({)ARBn—« | C(x) = A(x) B(x) 
k=0 
n na An 
binomial sums | C, = ¥ (Z)Ax C(x) = e*A(x) 
k=0 
marking Cn = nAp C(x) = xA"(x) 
scaling Cn =c"An C(x) = A(cx) 
Figure 4.4 


Basic relations between sequences and their generating functions. 


A generating function A(x) represents an analytic function for |x| < R, where 


-1 
R= (iimsup nl) 
n—0co 


denotes the radius of convergence. Thus, if R > 0 then we can either use a differen- 
tiation to represent the sequence 


or we use Cauchy’s formula 


1 dx 
A — A p= Take, 
"Oni [ (x) xrrl? 


where C is a closed curve inside the region of analyticity of A(x) with winding num- 
ber +1 around the origin. 

Another point of view on generating functions is that they can be considered as a 
power series generated by certain combinatorial objects. Let @ be a (countable) set 
of objects, for example, a set of graphs and 


|- ¢:67N 


a weight function that assigns to every element y € @ a weight or size |y|¢~. We 
assume that the sets 


G=\le Un) ={re@:lr 


e=n}  (n€N) 
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are all finite. Set C, = |@,|. Then the ordinary generating function C(x) of the pair 
(@,|-|w) that we also call combinatorial structure is given by 


C«)=) xlte — ys Ox 


YEE n>0 


and the exponential generating function C(x) by 


s xl x" 
Coy=). cae =O. 


ye6 ly ¢! n>0 n! 


The choice of ordinary generating functions or exponential generating functions de- 
pends on the kind of problem. As a rule unlabeled (or unordered) structures should 
be counted with the help of ordinary generating functions and labeled (or ordered) 
structures with exponential generating functions. 

For example, the ogf of binary sequences, where the weight denotes the length, 
is given by 


Nt 1 
B(x) = Y 2x" =. 


n>0 


An example of an egf is the egf of permutations of finite sets, where the weight 
is the size of the finite set: 


One major aspect in the use of generating functions is that certain combinatorial 
constructions have their counterparts in relations for the corresponding generating 
functions. 

We again use the notation <,¥4,@,... for sets of combinatorial objects with 
corresponding size functions | -|.7,|-|g,|-|¢v,.--. First suppose that the objects that 
we consider are (in some sense) unlabeled or unordered. We have the following basic 
operations: 


Disjoint union. If <& and Z are disjoint then @ = SW + B= AUG denotes the 
union of & and ¥. 


Cartesian product. @ = <&/ x & denotes the Cartesian product. The size function 
of a pair 7 = (0,8) is given by |7le = lll + [Blz. 


Sequence. Suppose that </ contains no object of size 0 and that the sets .o/, o/ x 
A,AX AX B,... are disjoint. Then 


6 =Seq(#):= {€} + P+ HxXA+AxXAXA+>>: 


is the set of (finite) sequences of elements of <% (€ denotes the empty object of size 
zero). Alternatively Seq(.%) is denoted by </*. 
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Powerset. Let @ = PSet(./) denote the set of all finite subsets of .. The size of a 
subset {041,02,..., 0} C & is given by |{01,02,..., 0} psetio) = |O1|v +|O2|a + 
“+ | Qos. 


Multiset. Let @ = MSet(./) denote the set of all finite multisets for)! ; ad”, oy a} 
of /, that is, the element @; is taken j; times (1 <i <k). Its size is given by 


[{or!, o?,..., 08 Feset(ar) = AlCl + i2lOaler +--+ jel Onley. 
Cycle. The cycle construction @ = Cyc(./) can be seen as (Seq(./) \ {€}) /S, 


where S is the equivalence relation that identifies two finite sequences (Q,..., &,) 
and (@;,...,@;) if a cyclic shift transfers them into each other. 


As already indicated these combinatorial constructions have counterparts in rela- 
tions of generating functions. Compare also with the relations in Figure 4.4: 


Theorem 4.3.2. The above mentioned combinatorial constructions imply the follow- 
ing relations for the corresponding generating functions: 


combinat. constr. | ogf 


C—A+B C(x) =A(x) + B(x) 
A(x) 


C=LXB C(x) =A(x)B(x) 

6 =Seq() C(x) = Tat 

€ =PSet(W/) | C(x) =exp|[ A(x) SAC?) SAGs) =: ) 

6 =MSet(#) | C(x) =exp A(x) + 5A (2) + 5A) + ) 
= _y 9h) ! 

€ =Cyc(#) Ca) = 2 7 8G AB 


(@(k) denotes the Euler totient function. ) 


Proof. We just sketch some parts of the proof. For example, the ogf of the Cartesian 
product @ = o& x & is given by 


C(x) = Y l(a) — y yal tla] — y lal. y x!l = A(x) - B(x). 


(a,b)EeLxXB (a,b)EeLxXB aca bEeB 


This implies that 1/(1—A(x)) represents the ogf of Seq(.”). 
The generating function of the finite subsets of / is given by 


C(x) = T] (1 +x!) =JJa+x). 


acd n>1 


By taking logarithms and expanding the series log(1 +”) one obtains the proposed 


290 Handbook of Enumerative Combinatorics 


representation C(x) = exp (A(x) — 5A(x?) + 4A(x) $---). Similarly the ogf of finite 
multisets is given by 


oo (s)-Bes) 


acd n>1 


Finally we mention that the relation for the cycle construction relies on Pélya’s 
theory of counting (see [13, p. 729]). a 


There are similar constructions of so-called labeled or ordered combinatorial ob- 
jects with corresponding relations for their exponential generating functions. We call 
a combinatorial object 7 of size n labeled if it is formally of the form y = Y x 2, where 
m € G, is a permutation. For example, we can think of a graph ¥ of n vertices and the 
permutation 7 represents a labeling of its vertices. We list some of the combinatorial 
constructions where we have to take care of the involved permutations. 


Disjoint union. If 2 and F are disjoint then @ = SW + B= AUF denotes the 
union of & and ¥. 


Labeled product. The labeled product @ = </ « & of two labeled structures is 
defined as follows. Suppose that a = & x 1 € of has size |a@| =k andB =BxoE B 
has size |b| = m. Then we define a « B as the set of objects ((@,B),7), where tT € 
Gxk4m runs over all permutations that are consistent with 2 and o in the following 
way: There is a partition {j1, j2,..., jx}, {41,€2,.--,€m} of {1,2,...,k +m} with jy, < 
Jo <c++ < jp and €) < ly < +++ < & such that 


T(1) = faci), T(2) = jncays---,T(k) = jn) and 
T(k+1) = o1), t(k+2)= 6(2)++09 Tk +m) = Lom). 


Finally we set 
AXB= U ax B. 
acd, BEB 


The size of ((&, B),7) is given by |((&,B),7)| = |a| +|B]. 
Figure 4.5 shows a small example for the labeled product. 


O-@ + MO = (O-@: @) (O-®: @) (@—-®: W)} 


Figure 4.5 
Labeled product. 
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Sequence. Suppose that ./ contains no object of size 0 and that the sets 7, «a, 
A «A * B,... are disjoint. Then 


6 =Seq(#):= {E} + H4+GH*AH+AxA «++: 
is the set of (finite labeled) sequences of elements of 2. 


Set. Similarly we define unordered labeled sequences (or a set) by 
1 1 
C =Set(#) SET pp eT a ES 


where the shorthand notation ia * Of «+++ * of means that we do not take care of 
the order of the n elements in the sequence A * AH «+++ * A. 


Cycle. We again define the cycle construction by @ = Cyc(/) = (Seq(.#) \ {€}) /S, 
where S is the equivalence relation that identifies two finite sequences when they can 
be transferred into each other by a cyclic shift. 


Theorem 4.3.3 The above mentioned combinatorial constructions for labeled ob- 
jects imply the following relations for the corresponding exponential generating 


functions: 


combinat. constr. | egf 


C=A+B C(x) = A(x) + B(x) 
C—-A+*B C(x) = A(x) B(x) 

i T 
See) ie 
 =Set(A) C(x) = exp ( () 
C= Cyc(#) C(x) = log 


Proof. |The only case that has to be explained is the labeled product @ = & « Z. 
If |a| =k and |B| =m then there are exactly eee) possible ways to partition 
{1,2,...,k +m} into two sets of size k and m. Thus a@ « B has size (8) and conse- 


quently 


C(x) = 


als lel Bl, 
Ges i Se Cor 
ac pea Il /(l@l+IB)! ge lel! gay lB! 


a 
Finally we mention that several notions and properties of the above discussed 


univariate generating functions can be extended to the multivariate case. For exam- 
ple, the underlying idea in the binary case is to consider (in addition to | - |v) a second 
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size function that we denote by p¢(-). Then the corresponding bivariate generating 
function is given by 


C(x,u) = y xlte ype™ — y Cag’ 
YEO n,k>0 
where Cr« = |{Y€ @ : |Yie =n, pe(y) = K}l- 
We just give here a simple example how the above mentioned combinatorial con- 


structions can be adapted. Let us consider @ = Seq(.#/) and let p¢(y) denote the 
number of ./-parts of y € @. Then we have 


- 1 
~ 1—uA(x)’ 


We will use this kind of extension in several enumeration problems below. 


C(x,u) = 1+4+uA(x) +u7A(x)? +--: 


4.3.2 The Lagrange inversion formula 


Let A(x) = YysoAnx" be a power series with Ag = 0 and A; 4 0. The Lagrange 

inversion formula provides an explicit representation of the coefficients of the inverse 

power series Al“! (x), which is defined by A(A!“!I(x)) = Al“! (A(x)) =x. 

Theorem 4.3.4 Let A(x) = Y, A,x" be a formal power series with Ay =0 and A, # 
n>0 


0. Let A“! (x) be the inverse power series and g(x) an arbitrary power series. Then 
the n-th coefficient of g(Al“'](x)) is given by 


relay) =“ Ney (se) wea), 


In tree enumeration problems the following variant is more appropriate. 


Theorem 4.3.5 Let B(x) be a power series with (0) 4 0 and Y(x) the (unique) 
power series solution of the equation 


Y (x) =x®(Y(x)). 


Then Y (x) is invertible and the n-th coefficient of g(¥ (x)) (where g(x) is an arbitrary 
power series) is given by 
1 


b"le¥@)) == [e's @W)" (x2 1), 


Theorems 4.3.4 and 4.3.5 are equivalent. If A(x) = 
where Y (x) satisfies the equation Y (x) = x®(Y (x)). * 

As an example we consider the equation Y (x) = x(1+Y(x))? that is related to the 
enumeration of binary trees (see Section 4.4.1). Here we have ®(x) = (1 +x)? and, 
thus, we obtain 


Yy = [x"]¥ (x) = vlut-+up" = 2 a ) = er 


n\n-1 


x/®(x) then Al-!I(x) = Y(x), 


*A proof of Theorem 4.3.5 can be found in Chapter 2 on Analytic Methods. 
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4.3.3 The dissymmetry theorem 


The enumeration of rooted objects (like rooted trees) is usually easier than the enu- 
meration of unrooted objects. Nevertheless there is a general method to relate un- 
rooted and rooted objects by the so-called Dissymmetry theorem. It was originally 
observed by Otter [22] and generalized by Bergeron, Labele and Leroux [3]. 


Theorem 4.3.6 Let </ be aclass of trees, f° the corresponding class of trees, where 
a node is marked, &°~° the class, where an edge is marked, and @°*° the class, 
where an edge is marked and is given a direction. Then there is a bijection between 


A+aA°° and L+Hh°° 


Proof. Recall that the center of a tree is either a single node, the central node, or a 
pair of adjacent vertices, the central edge. (The center is defined as the set of nodes 
of a tree, where the maximal distance to other nodes is minimal.) This means that an 
unrooted tree has either a distinguished root or a distinguished edge. Consequently 
we can see the class 

oe" = (f° +4 of°~°) \ A 


as the class of trees that are either rooted at a non-central node or at a non-central 
edge. Therefore it is sufficient to formulate a bijection between ./” and °°. 

We distinguish between two cases. If a tree in </”° is marked at a vertex vo that 
is different from its central node v. (when the center is a node) let v; be the vertex 
adjacent to vg on the unique path from vo to the center (that contains the central edge, 
too, if the center is an edge). Now we mark the tree at the directed edge from vg to 
vy, and obtain a tree in &°°. Second, if a tree in </”"° is marked at an edge eg that 
is different from its central edge e, (when the center is an edge) then orient e9 away 
from the center and we obtain a tree in. °°. 

It is easy to see that this mapping is actually a bijection. | 


As an example we apply the Dissymmetry theorem for the class of unlabeled trees 
A = F (see Section 4.4.4). Then «° is in the class of rooted unlabeled trees .7, 
which has generating function T(x). The class </°’° can be identified with 7 x 7, 
the class of ordered pairs of rooted trees; we can join two rooted trees by a directed 
edge from the root of the first tree to the root of the second tree. Clearly the generating 
function of /°~° is T(x)’. Finally, the class .o/°~° can be seen, too, as a pair of 
rooted trees, however, as unordered pairs. In the corresponding counting problem we 
have to distinguish between the case, where the two rooted trees are different and 
where they are the same. In the first case we can identify an unordered pair {7 , 7>} 
with the two (different) ordered pairs (T,72),(7I2,7)), whereas in the second case 
we just have one ordered pair (T,,7,). This implies that the generating functions 
corresponding to «/°~° equals 4 (T(x)? +T(x?)) since T(x?) counts pairs of the 
form (71,71). Summing up it follows that 


T (x) +T (x)? =T(x)+ ; (T(x)? +T (x*)) 
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or 


which is Otter’s theorem [22]. 


4.3.4 Asymptotics 


We mention here only a theorem (by Meir and Moon [20]) that is related to the 
Lagrange inversion and, of course, to tree enumeration problems. * 


Theorem 4.3.7 Let R denote the radius of convergence of a power series B(x) = 
Yio xk with non-negative coefficients and let Y(x) = Ly>1Ynx" be the solution 
of the equation Y (x) = x®(¥ (x)). Suppose that there exists t with 0 < t < R that 
satisfies TH'(t) = (Tt). Set d = ged{j > 0: pj > 0}. Then 


Y, =d sees ee (1+0(0-')) (n = 1 mod d) 


and Y, =0 ifn #1 mod d. 


Note that in the most frequent case d = 1 the condition n = 1 mod | is satisfied for 
all integers n (and the case n = 1 mod 1 does not appear). 

This theorem applies to several classes of trees, in particular to simply generated 
trees that will be discussed in Section 4.4.5. So it is quite universal. 

As an example let us discuss the equation Y(x) = x/(1 —Y(x)) that is related 
to the counting problem of planted plane trees (see Section 4.4.2). Here B(x) = 
1/(1 —x) and the equation t&’(t) = (7) has the solution t = 1/2. We have d = 1, 
®'(t) = 4, b(t) = 16, and consequently Y,, = [x”] ¥ (x) is asymptotically given by 


n—-1 


Jrn3/2° 


In Section 4.4.2 we will see that Y, = ewe! and it is easy to check by Stirling’s 


formula that this asymptotic expansion is correct. 

It should be mentioned, too, that Theorem 4.3.7 can be generalized (or adjusted) 
in severals ways that are relevant for tree counting problems. Therefore we discuss 
the background of Theorem 4.3.7 a little. The main observation from the analytic 
point of view is that the solution Y(x) of the implicit equation Y = x®(Y) has a 
so-called squareroot singularity at x9 = t/®(t), which means that there is a local 
expansion of the form 


Yn 


Y (x) = ay +a, V/x9 —X + a2(x9 —x) +43(xo — x)? ++ ‘ 


that converges for |x —xo| <n (for some 7 > 0) and where apg = T and a; = 
—/2(t)/(xo®"(t)) < 0. If xo is the only singularity on the circle of convergence 


*Tt is also discussed in Chapter 2 on Analytic Methods. 
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|x| = xo, which is easy to check, then it follows by the transfer theorem by Flajolet 
and Odlyzko [12] that Y, is asymptotically given by 


AV/X0 See 
a — ei n 240 (xq"n a) : 
A similar procedure applies if we consider a function of the form C(x) = g(¥(x)), 
where g(x) is regular for x = T = Y (xo). Then a local expansion shows that C(x) can 
be represented similarly: 


C(x) = co +¢1VX0 —X + €2(x9 — x) +.€3 (x0 xi? free, 


where co = g(T) and c; = g’(T)ay. If c, £0 then we are in precisely the same situ- 
ation as above. However, if c) = 0 and c3 4 0 then the singular term (xo ee 2 is 
dominating and we obtain an asymptotic expansion of the form 


3/2 
i 33% in ayes 
Ch = [x"]C(x) = JE Xn 2 +0 (x5 n 72), 


4.4 Unlabeled trees 


As mentioned above, generating functions are quite natural in the context of tree 
counting since (rooted) trees have a recursive structure that usually translates into 
recurrence relations for corresponding counting problems. In what follows we will 
count several classes of unlabeled trees with the help of generating functions. We 
start with binary trees, which are probably the most basic ones. 


4.4.1 Binary trees 


Binary trees are planted rooted trees, where each node is either a leaf or it has two 
distinguishable successors: the left successor and the right successor. The leaves of 
a binary tree are usually called external nodes and those nodes with two successors 
internal nodes. It is an easy exercise to show that a binary tree with n internal nodes 
has n+ 1 external nodes. Thus, the total number of nodes is always odd. 


Theorem 4.4.1 The number B,, of binary trees with n internal nodes is given by the 
Catalan number 
B= 1 /2n 
7 nt liny 


Proof. Since this is the first example, we will provide several variants of the proof. 
However, all of them all related to generating functions. 

The first variant is the most basic one that sets up a recurrence that is then solved 
with the help of generating functions. Suppose that a binary tree has n + | internal 
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Figure 4.6 
A binary tree. 


nodes. Then the left and right subtrees are also binary trees (with k and n — k internal 
nodes, where 0 < k <n). Thus, one gets directly a recurrence for the corresponding 
numbers: 


n 
Bnui = ¥ ByBn—k- (4.1) 
k=0 


The initial value is By = | (where the tree consists just of the root that is an external 
node). 
This recurrence can be solved using the generating function 


B(x) = Yo Bpx". 


n>0 
By the properties of generating functions we find the relation 
B(x) =1+4xB(x)? (4.2) 
and consequently an explicit representation of the form 
1—-J1—4x 
2x ; 


(Note that the other solution (1 + /1—4x/(2x) of the quadratic equation does not 
represent a power Series, since it is singular at x = 0.) Hence we obtain 


ea a Oe 1 


1/7 4 1 (/2n 
ene? 2 —4yrrl _ 
staat ) ailn): 


By inspecting this first variant of the proof of Theorem 4.4.1 one observes that 
the recurrence relation (4.1), together with its initial condition, is exactly a translation 
of a recursive description of binary trees: 


B(x) = 


B, = [x"] 
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A binary tree # is either just an external node or an internal node (the 
root) with two subtrees that are again binary trees. 


Of course this can be rewritten in terms of combinatorial constructions: 


B=O+0x Bx B=O+0x B’, (4.3) 


where & denotes the system of binary trees; LK) represents an external and o an inter- 
nal node. This leads to a corresponding relation (4.2) for the generating function (see 
Theorem 4.3.2): 

B(x) =1+4xB(x)’, 


where the size of a binary tree is the number of internal nodes. See also the schematic 
Figure 4.7. Thus, setting up the equation (4.2) for the generating is just a translation 
of the combinatorial construction. 


Figure 4.7 
Recursive structure of binary trees. 


In order to obtain a formula for the coefficient B,, = [x”] B(x) we can either use the 
above procedure (explicit solution of the quadratic equation and using the binomial 
series) or we just apply the Lagrange inversion formula. 

By setting B(x) = B(x) — 1 the equation (4.2) rewrites to 


Now we can apply Lagrange’s inversion formula (Theorem 4.3.5) with ®(x) = (1+ 
x)? and obtain (for n > 1) 


see also Section 4.3.2. | 
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2n 
n 


The explicit formula for B, = = ( 
expansion. By applying Stirling’s formula n! ~ /2ane~"n" we obtain 


) can also be used to obtain an asymptotic 


qr 
Vani? 


This is, of course, consistent with Theorem 4.3.7 applied to ®(x) = (1 +x)’. 


Bn ~ 


A direct generalization of binary trees are m-ary rooted trees, where m > 2 is a 
fixed integer. As in the binary case (m = 2) we just take into account the number n 
of internal nodes. The number of leaves is then given by (m—1)n+1 and the total 
number of nodes by mn + 1. 


Theorem 4.4.2 The number Bw”) of m-ary trees (m > 2) with n internal nodes is 


given by 
Bi”) = 1 mn 
(m—1)n+1\n 


Proof. |The combinatorial decomposition shows that the generating function satis- 
fies B’”) (x) = 1+xB"”) (x). Consequently the function B™ (x) = B(” (x) — 1 satis- 
fies B™ (x) = x(1 +B) (x))” and, thus, by the Lagrange inversion formula (Theo- 
rem 4.3.5) we get forn > 1 


Bo") = [eB (x) = ace +u)"™ = -( oe ) 2 note ): 


n—-1 m—1 
| 


Another variant of binary trees are Motzkin trees that are ordered rooted trees, 
where each node has either no, or one, or two successors (recall that ordered means 
that successors of each node are equipped with a left-to-right-order). In this context 
it is common to take the number or edges as the size of a Motzkin tree. Then the 
generating function M(x) of Motzkin trees satisfies the equation 


M(x) =1+xM(x)+x°M(x)?. 


By using the substitution T(x) = xM(x) we get T(x) = x(1 +7 (x) + T(x)’). Hence, 
the Motzkin numbers are given by 


My = b"|M(x) = T(x) = 


= ae [u)(1 Sage 


There is no simple formula for M,. Nevertheless, they can be expressed as hyperge- 
ometric sums. For example we have 


1 1 
Mn =2F\ 5 —n), 5m 24 ; 
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(wea 


Figure 4.8 
Planted plane tree. 


where 9F} (a,b;c;z) =Lnso ae z is the usual hypergeometric function. When we 
apply Theorem 4.3.7 to the function U (x) = xM(x) and to ®(x) = 1+x+.x? we also 


obtain the asymptotic formula 
| 3 
M,~ sary grr : 
4nn3 


4.4.2 Planted plane trees 


Another interesting class of trees are planted plane trees. Sometimes they are also 
called Catalan trees. Planted plane trees are again rooted trees, where each node 
has an arbitrary number of successors with a natural left-to-right-order (this again 
means that we are considering plane or oriented trees). The term planted comes from 
the interpretation that the root is connected (or planted) to an additional phantom 
node that is not taken into account (see Figure 4.8). Usually we will not even depict 
this additional node when we deal with planted trees. However, it is quite useful to 
define the degree of the root r by d(r) = d*(r) +1, which means that the additional 
(planted) node is considered a neighbor node. This has the advantage that in this case 
all nodes have the property d(v) = d*(v) +1. 


Theorem 4.4.3 The numbers P, of planted plane trees with n > 1 nodes are given 
by 
1 & - ‘ 
Py = - : 
n\n-1 


Proof. Let Y denote the set of planted plane trees, where the size is the number of 
nodes. The combinatorial decomposition Y = {o} x Seq(#) implies 
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ALO 


Figure 4.9 
Recursive structure of a planted plane tree. 


for the corresponding generating function P(x), see also the schematic Figure 4.9. 
Consequently 


P= bP) = twa —wr* = au ( =n _ a eae 


n n n—-1 n\n-1 


Alternatively we have P(x) = x -+P(x)* and consequently P(x) = xB(x), which is 
equivalent to P, = B,_1. | | 


The property that the number of planted plane trees of size n equals the (n — 1)th 
Catalan number explains the term Catalan tree. 

It is worth mentioning that the relation P, = B,_; has a natural interpretation. 
First there is a natural bijection between planted plane trees with n nodes and binary 
trees with n — | internal nodes, the so-called rotation correspondence. Let us start 
with a planted plane tree with n nodes and apply the following procedure: 


1. Delete the root and all edges going to the root. 


2. If a node has successors delete all edges to these successors despite one edge 
to the leftmost one. 


3. Join all these (previous) successors with a path (by horizontal edges). 
4. Rotate all these new (horizontal) edges by the angle 2/4 below. 


5. The remaining n— | nodes are now considered as internal nodes of a binary 
tree. Append the (missing) 7 + 1 external leaves. 


The result is a binary tree with n — 1 internal nodes. It is easy to verify that this 
procedure is bijective (compare with the example given in Figure 4.10). 

A second interpretation of the relation P, = B,-; comes from an alternate re- 
cursive description of planted plane trees. If a planted plane tree has more than one 
node then we can delete the left-most edge of the root and obtain two planted plane 
trees, the original one minus the left-most subtree of the root and the left-most sub- 
tree of the root (see Figure 4.11). Obviously this description leads to the recursive 
description 


Pa=atrFKf’, 


Kh NA LE. 
‘eR 
we 


Figure 4.10 
Rotation correspondence. 


Figure 4.11 


Alternative recurrence for planted plane trees. 
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x(i) 


~~ 


Figure 4.12 
Depth-first search of a rooted tree and the corresponding Dyck path. 


which gives 
P(x) =x+P(x)’. 


We already observed that by setting P(x) = xB(x) we get B(x) = 1 +xB(x)* and so 
P, = Bn—1. Note that the recursive description (depicted in Figure 4.11) also leads to 
the rotation correspondence. 

Planted plane trees are also in bijection to Dyck paths, which are lattice paths on 
the integer points that start at (0,0), use only steps of the form (1,1) and (—1,—1) 
and terminate at a point (27,0) on the x-axis (for some n > 0) so that the path stays 
always above the x-axis. The right part of Figure 4.12 shows an example for n = 4. 

The bijection can be stated in terms of the so-called depth-first search. For the 
sake of consistency we assume that the planted plane tree has n edges (or n+ 1 
vertices; the planted vertex is not taken into account). The depth-first search can be 
described as a walk (v(i),0 < i < 2m) around the vertices, where v(0) = v(2n) is 
the root. We proceed inductively. Given v(i) choose (if possible) the first edge (in the 
natural order) incident to v(i) leading away from the root that has not already been 
used, and let (v(i),v(i+ 1)) be that edge. If all edges leading away from v(i) have 
been already used let (v(i),v(i+ 1)) be the edge from v(i) leading toward the root. 
This walk terminates with v(2n), which is (again) the root. 

The search depth x(i) is now defined as the distance of v(i) to the root. For non- 
integer i we use linear interpolation and thus x(t), 0 <t < 2n, can be identified with 
a Dyck path of 2n steps (see Figure 4.12). 

Obviously this leads to a bijection between planted plane trees with n+ 1 vertices 
and Dyck paths of length 27. Thus the number of Dyck paths of length 2n is again 


given by the Catalan number —. leel This is a very well-known fact (and there are 


several different proofs for die tonal), 

In contrast to binary (or m-ary trees) the number of leaves in a planted plane 
tree is not related to the total number of leaves. Actually we can refine the counting 
problem and also get an explicit formula for the number of planted plane trees with 


a given number of leaves. 
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Theorem 4.4.4 The numbers P,,, of planted plane trees of size n with k leaves are 
given by the Narayana numbers 


nt) 


Proof. Let P(x,u) =Yyx Par denote the bivariate generating function of the 
numbers PF, ;. Then, following the recursive description of planted plane trees, one 
gets 

xP(x,u) 


P(x, u) =xu+xY p(x,u)* = xu+ Chea 


k>1 


For instance, let x be considered as a parameter. Then we have 


and consequently by applying Lagrange’s inversion formula for the variable u we 
obtain 


Finally this implies 


Sle re ee 


It is interesting to discuss the asymptotic behavior of P, x. By using Stirling’s 
formula we obtain a bivariate asymptotic expansion for P, , of the form 


Put = sain) (2) (+0(7) +03) 
#) (ra) (+2(@) +e) 
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where we assume that k + o and n—k — oo, In particular, if we fix n then P,,, is 
maximal if k ~n/2 and we locally get a behavior of the kind 


4” =2h)2 
Pig — exp ( = ) (4.4) 
, wn 


n 


if |k —n/2| < C\/n (for an arbitrary constant C > 0). This approximation has several 
implications. First, it shows that it is most likely that a typical tree of size n has 
approximately n/2 leaves and the distribution of the number of leaves around n/2 
looks like a Gaussian distribution. 

We can make this observation more precise. Let n be given and assume that each 
of the P,, planted plane trees of size n is equally likely. Then the number of leaves is 
a random variable on this set of trees, which we will denote by X,,. More precisely, 
we have i 

nk 
P{X, =k} = a 


n 


Then EX, ~n/2 and VarX,, ~ n/8, and (4.4) can be restated as a weak limit theorem: 


— N(0,1). 


Actually there are lots of statements of this kind and the method of generating func- 
tions can be used to prove them directly (for more details see Chapter 3 on Asymp- 
totic Normality in Enumeration or [13, 11]). 


4.4.3. Unlabeled plane trees 


A tree usually has several different embeddings into the plane. Planted plane trees, in 
particular, take into account all possible planar embeddings of rooted trees into the 
half plane, where the root is at the boundary of the half plane. With the help of this 
observation and by using the cycle construction and the Dissymmetry theorem it is 
possible to derive from planted plane trees the structure of plane trees Y, that is, all 
different embeddings of (unlabeled) trees into the plane. 


Theorem 4.4.5 The generating function P(x) of plane trees FP is given by 


1 1 
P(xk) 2 


5 p(k) 
P(x) =x+x log 
» k Le 


where P(x) = (1— V1 — 4x) /2 denotes the generating function of planted plane trees. 
The numbers P, = {x"|P(x) (that count different embeddings of (unrooted) trees of 
size n into the plane) are asymptotically given by 


ee nea - 
Py = satin */? (1+0(n7!)). (4.5) 
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Proof. First, rooted plane trees # can be described as 4 = 0+0 x Cyc(W): A 
rooted plane tree it is either just the root (when it is of size 1) or it is the root attached 
with a cyclically ordered sequence of planted plane trees. 

Hence, the generating function R(x) is given by 


1 
P(xk)’ 


p(k) 
R(x) =x+x log 
d k 1 


Second, the Dissymmetry theorem (Theorem 4.3.6)) implies 


P(x) = R(x) — ; (P@)? — P(x) : 


We just have to mimic the arguments for unlabeled trees following the proof of The- 
orem 4.3.6. 

In order to obtain the asymptotics we observe that P(x*) is analytic at x9 = 1/4 
for all k > 2, and thus, only the leading term 1/(1 — P(x)) (corresponding to k = 1) 
has to be taken into account in order to obtain the local behavior of P(x) around 
xo = 1/4. By using the explicit form of P(x) = (1 — 1 — 4x) /2 it follows that P(x) 
has a local expansion of the form 


P(x) =co+ca(1 4x) + 2(1 ax) /2 4... 


which means that the term corresponding to /1—4x cancels. Hence we obtain (4.5); 
compare with Section 4.3.4. a 


4.4.4 General unlabeled trees 


The counting problem of unlabeled unrooted trees . is actually a quite involved 
one. Here we do not care about the possible embeddings into the plane. We just think 
of trees in the graph-theoretical sense. In addition to unrooted trees we consider—as 
usual—also the class of rooted unlabeled trees 7. 

Let us denote by T, and T,, the corresponding numbers of those trees of size n, 
for example we have 


T; = 1, 1 1, PT 1, T%=2 and 7; 1h=1,%=2,%=4. 


We already observed in Section 4.3.3 that the Dissymmetry theorem implies a 
relation between the generating function of rooted and unrooted trees: 
P(x) =T(a)— 57 (x) +57 yy (4.6) 
Thus we only have to get some relation for rooted trees. Of course, they can be 
recursively defined, however, instead of a sequence of trees we have a multiset of 
trees attached to the root: 


TF =oxMSet(Z). 
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Thus, the generating function T(x) satisfies 


T (x) =xexp (re SP (x) aT’) ~), (4.7) 


Of course, the equation (4.7) can be transformed into a recurrence 


n 


1 
Ti=-) Yd Th j+1, 
” j=1dlj 


and an even simpler relation holds between 7,, and T,,. However, it seems that there 
is no proper explicit formula for T,, and 7,,. Nevertheless there are asymptotic expan- 
sions for them and by using extensions of the mentioned counting procedure it is also 
possible to study several shape characteristics of these kinds of trees. 

Historically the relation (4.7) was already known to Pélya as an application of 
Polya’s theory of counting [23]. The amazing relation (4.6) is due to Otter [22]. 
More precisely Pélya [23] already discussed analytic properties of the generating 
function T (x) and showed that the radius of convergence p satisfies 0 < p < 1 and 
that x = p¢ is the only singularity on the circle of convergence |x| = p. Later Otter [22] 
showed that T(p) = 1 and used the representation (4.8) to deduce the asymptotics for 
T,. He also calculated p + 0.338219. However, his main contribution was to show 
(4.6). Consequently he derived (4.9) and (4.11). 


Theorem 4.4.6 The generating functions T (x) and T (x) satisfy the functional equa- 
tions (4.7) and (4.6) and have a common radius of convergence p © 0.338219, which 
is given by the relation T(p) = 1. Moreover they have a local expansion of the form 


T (x) =1—b(p —x)'? +c(p —x) +d(p—x)7/? +O((p—x)?)) (4.8) 


2 nel PV) 
Py = tte)? BP tp x) +be(p —x)?/*+0((p—x)’), (4.9) 


where b © 2.6811266 and c = b*/3 = 2.3961466 and x = p is the only singularity 
on the circle of convergence |x| = p. Finally, T, and T, are asymptotically given by 


_ VP —3/2,.—n -1 
In = 37g" p "(1+0(n"')) (4.10) 


and 


. b3p3/2 
tT, = —S=n 
4/1 


We do not give the proof here. We just mention that it follows the same lines as 
the proof of Theorem 4.4.5. In a first step it is shown that T(x) has a local expansion 
of the form (4.8), which follows from a proper extension of Theorem 4.3.7. This 
provides the asymptotic expansion (4.10) for 7;,. Finally by using (4.8) and (4.6) the 


“5 (14+0(n-!)). (4.11) 
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local expansion (4.9) for T(x) follows and the asymptotic expansion for 7, is an 
immediate consequence. 

In a similar way we can deal with unlabeled binary trees 4, where we do not care 
about the embedding in the plane. We only require that every node has outdegree 0 
or 2 and do not care about the order of the subtrees. The corresponding unrooted 
version & is the set of unlabeled trees, where every node has either degree 1 or 3 (if 
it is not only a single vertex). Let the corresponding cardinalities of these trees (of 
size n) be denoted by B,, and B,, and the generating functions by 


B(x) = y B,x" and B(x) = y Byx". 


n>1 n>1 


Then we have (similarly to the above, compare also with [7]): 


Theorem 4.4.7 The generating functions B(x) and B(x) satisfy the functional equa- 
tions 


B(x) =x+ 5 (B(x)? + B(x?) (4.12) 
and 
B(x) =x+ , (B(x)? + 3B(x)B(x?) + 2B(x3)) — 5B)? + 5B). (4.13) 


They have a common radius of convergence Pz * 0.6345845127 which is given by 
P2 B(p2) = 1 and singular expansions corresponding to (4.8) and (4.9). Furthermore, 


B, and By, are asymptotically given by 
B, =cyn*?p5"(1+O0(n-!)) (n=1 mod 2) 
and 
B, = con! px” (1 +0(n7!)) 
with certain positive constants C1, C2. 

We just comment on the combinatorial part, that is, on (4.12) and (4.13). The first 
equation comes from the fact that rooted binary trees of this kind can be recursively 
defined as a single (root) vertex or the root followed by an unordered pair of binary 
trees. 

With the help of B(x) we obtain the generating function R(x) of rooted (unla- 


beled) trees, where all vertices (including the root) have either degree | or 3, that is, 
a root followed by an unordered triple of binary trees: 


R(x) =x+ 7 (B(x) +3B(x)B(x?) +2B(x3)). 
Finally, we apply the Dissymmetry theorem and obtain 


B(x) = R(x) — 5Blx)? +5B(0), 


This completes the proof of (4.13). 
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The proof of the analytic part runs along the same lines as in the proof of Theo- 
rem 4.4.5. 

As mentioned in the Introduction, the book by Bergeron, Labele, and Leroux [3] 
provides a systematic treatment of the counting problem of unlabeled trees and tree- 
like structures that is based on Pélya’s theory of counting, and many other things. 
The above two examples are only the tip of the iceberg. 


4.4.5 Simply generated trees and Galton- Watson trees 


Simply generated trees are weighted versions of rooted trees and have been intro- 
duced by Meir and Moon [20]. The idea is to put a weight to a rooted tree according 
to its degree distribution. 

Let ;, j = 0, be a sequence of non-negative real numbers, called the weight 
sequence. Usually one assumes that @p > 0 and @; > 0 for some j > 2. We then 
define the weight @(T) of a finite rooted ordered tree T by 


o(T)= [] ga+) =T1 9", 


veV(T) 720 


where d* (v) denotes the out-degree of the vertex v (or the number of successors) and 
D(T) the number of nodes in T with j successors. The numbers 


¥,= Y o(7) 


|T|=n 


are then the weighted numbers of trees of size n. 
It is convenient to introduce the generating series 


D(x) = G+ Gxt Qo? +++ = Y Qjx!. 


J20 


Theorem 4.4.8 The generating function Y(x) = Lys 1Ynx" of a simply generated 
family of trees satisfies 
¥(x) =x6(¥(x)) 


so that Y, is given by Y, = +[u""|(u)". 


Proof. Similarly to binary trees or planted plane trees we can write a relation for 
the weighted combinatorial objects of simply generated trees: 


Y = M04 Q1:0XY¥4+Q):0xYW7 +>, 
In terms of the generating function we, thus, obtain 


Y (x) =x@p +.x@1Y (x) +x@oY (x)? +--+: =x@(¥ (x). 
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Simply generated trees generalize several of the above examples of combinatorial 
trees. 

If oj =1 for all j > 0, that is, ®(x) = 1/(1 —x), then all planted plane trees have 
weight @(T) = | and Y,, is the number of planted plane trees. 

Binary trees (counted according to their internal nodes) are also covered by this 
approach. If we set @p = 1, | = 2, 2 = 1, and gj = 0 for j > 3, that is, B(x) = 
(1 +x)’, then nodes with one successor get weight 2. This takes into account that 
binary trees (where external nodes are disregarded) have two kinds of nodes with 
one successor, namely those with a left branch but no right branch and those with a 
right branch but no left branch. Similarly, m-ary trees are covered with the help of 
the weights @; = ("') or with ®(x) = (14x). 

If Qo = 91 = 2 = 1 and g; =0 for j > 3 or B(x) = 1+x+.’, then we recover 
Motzkin trees. Here only rooted trees, where all nodes have less than 3 successors, 
get (a non-zero) weight @(T) = 1: Y, is then the number M,,_; of Motzkin trees with 
n nodes (or n — | edges). 

If we set p; = 1/j! then 


nl-Y, =n"! 


denotes precisely the number of labeled rooted non-plane trees, see Section 4.5.1. 
The weight @; = 1//! disregards all possible orderings of the successors of a vertex 
of out-degree j and the factor n! corresponds to all possible labelings of n nodes. 
Hence, every labeled tree gets the same weight. 

We also note that Theorem 4.3.7 provides an asymptotic formula for Y,, for a wide 
class of simply generated families of trees (we only have to ensure the existence of 
T <R that satisfies tH’(t) = P(t). 

There is an intimate relation between simply generated trees and Galton- Watson 
branching processes. Let € be a non-negative integer-valued random variable, the 
so-called offspring distribution. The Galton-Watson branching process starts with a 
single individual (generation 0); each individual has a number of children distributed 
as independent copies of &. 

It is clear that Galton-Watson branching processes can be represented by ordered 
(finite or infinite) rooted trees T. We denote by v(T) the probability that a specific 
tree T occurs. We assume that P{§ = 0} > 0 so that there is positive probability that 
the resulting tree T is finite. We also mention an important parameter for Galton- 
Watson processes, namely the value E€, which is the expected offspring and governs 
the overall behavior of the process. The most interesting case is the case EG = 1 
which means that the average number of children is 1. This case is called critical. 

The generating function Y (x) = Y),+) Y,x” of the numbers 


¥,=P{\T|=n}= ¥, v(r) 


|T|=n 


satisfies the functional equation 
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where 


O(t) =Er = V° git! 


j20 


with g; = P{§ = j}. Observe that 


v(T) = We” =@(T). 


j20 


The weight of T is now the probability of 7. Thus Galton-Watson trees (conditioned 
on the size n) can be considered as special cases of simply generated trees. 

The converse is almost true. It is possible to scale a general simply generated 
family properly so that it mimics a Galton-Watson process with the property that 
(conditioned on 7) they are equivalent. The reason is the following one. If we replace 
9; by $; = ab/;, which is the same as replacing (x) by &(x) = a®(bx) for two 
numbers a,b > 0, then @(T) is replaced by 


&(T) =] (abi)?! =aTb7 107). 
j20 


Note that ) ; jDj(T) =|T| — 1. Hence, Y, = a"b"~'Y,,. In particular by setting b = 7, 
where T is the solution of the equation t&’(t) = (7) (if it exists) and a = 1/@(b) 
then we have 

@(1)=1 and @(1)=1. 


This means that (x) can be interpreted as the probability distribution of a discrete 
random variable with expected value 1, that is, the corresponding Galton-Watson 
process is critical. 

If we consider just trees T of size n then all weights satisfy @(T) = a"b"~!(T) 
which means that the distribution of the weights is the same. More precisely if we 
define the probability 2,(7) of a tree of size n by @(T)/Y, then these probabilities 
do not change by introducing these scaling constants a,b. 


4.5 Labeled trees 


We call a tree labeled if the vertices v are mapped to integer (or other) values ¢(v), 
the labels, and we distinguish two trees to be different if they have different labels 
even if the underlying unlabeled trees are the same. Of course the counting problem 
depends on the kind of labels, the conditions on the labels and on other properties of 
the trees (if we consider planted trees or not, etc.). In what follows we will consider 
some of the most interesting versions of labeled trees (partly taken from Moon’s book 
[21], where many more results can be found). We start with the famous formula by 
Cayley. 
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4.5.1 Cayley trees and related trees 


A Cayley tree of size n is a labeled tree, where the n nodes are labeled by 1,2,...,n 
and where we do not take care on embeddings into the plane. In labeled trees there is 
an easy relation between rooted and unrooted trees. Since there are exactly n different 
ways to make an unrooted tree to a rooted one by choosing one of the labeled nodes, 
the number of rooted labeled trees of size n equals the number of unrooted labeled 
trees exactly n times. Consequently it is sufficient to consider rooted labeled trees 
which has the advantage that one can use the recursive structure. 


Theorem 4.5.1 The number T,, of rooted labeled trees of size n is given by 
T,=n'!,~ 

Consequently the number of unrooted labeled trees of size n equals T, = n"~?. 
Proof. Let Y denote the set of labeled rooted trees. Then 7 can be recursively 
described as a root followed by an unordered k-tuple of labeled rooted trees for some 
k>0: 

TF =oxSet(Z). 
Thus, it is appropriate to use the exponential generating function T(x) = 
Ln>o Inx"/n! of T,. The above recursive description is then translated into 


T (x) =xe™), 


Finally, by Lagrange’s inversion formula (Theorem 4.3.4) 


pe = yn! 


1 
T, =n!—[u 
n 


2 


We remark that the generating function T(x) of the number 7, = n”~? can be ex- 


pressed as 


This follows either by observing that the derivative of T(x) — T(x)*/2 is given 
by T!(x) —T(x)T'(x) = T(x)/x (since T(x) = xe? implies T’(x) = T(x)/(x(1 — 
T (x)))). Hence 


x)2 x n-1 xh . 
r(x) 78 =| En at = Fn" = TH), 


n>1 : n>1 


Alternatively we can use the Dissymmetry theorem (Theorem 4.3.6) to express T (x) 
in terms of T(x). 


There are several other proofs of Cayley’s formula. We just mention two further 
ones. One is based on the interesting fact that the number of trees of size n, where 
the vertex with label j has degree d;, 1 < j <n, is given by the number (see [21]) 


n—2 
d,—1,...,d,—-1/° 


312 Handbook of Enumerative Combinatorics 


Equivalently we have 


y ge Bel eget! — (xy Se nee oe (4.14) 


T:|T|=n 
where the sum is taken over all labeled trees of size n and where v;(T) denotes the 
vertex with label j. Of course by setting x) = x2 =--- = x, = 1 we recover Cayley’s 
formula 7, = n"~?. 

Let C,(x1,.-.,%,) denote the left-hand side of (4.14). Then C, is a symmet- 
ric and homogeneous polynomial of degree n — 2. The specialized polynomial 
Cn(x1,---;Xn-1,0) corresponds to those trees with d(v,(T)) = 1. This means that 
Vy is a leaf and is linked to a vertex v; with 1 < j <n. By removing v, and the edge 
to v; we are left with a labeled tree of size n — 1. By rephrasing this procedure in 
term of C,, we get 


Cal Mater ight O) = reo a Ca Raat) 


and, thus, it follows by induction that 

Cn(x1,---;Xn—1,0) = (1 Seege aye: 
Since the degree of C,, equals n — 2, which is smaller than n, it is possible to recon- 
struct Cy(*1,...,Xn) from T (x1,...,%n—1,0). We just have to complete the appearing 
monomials in C,(x1,...,Xn—1,0) to make them symmetric in x),...,%n. Thus, it fol- 
lows that Cy(x1,...,%n) = (41 ++++ +.xn)"-?. 

A further and completely different way for computing 7, = n"~? is to observe 
that an unrooted labeled tree can be interpreted as a spanning tree of the complete 
graph K,, with nodes 1,2,...,n (see Figure 4.13). Thus, the matrix-tree theorem (The- 
orem 4.6.1) can be applied. (We will discuss spanning trees of a graph and how to 
compute their number in more detail in Section 4.6.1.) 


Figure 4.13 
Two of 16 possible spanning trees of K4. 


If we distinguish also the number of leaves in rooted labeled trees we obtain 
a formula in terms of the Stirling numbers of the second kind S,,;, which denote 
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the number of partitions of a set of size n into k parts. Alternatively, these Stirling 
numbers can be defined by 


or explicitly by 


Theorem 4.5.2 The number T,,, of rooted labeled trees of size n with k leaves is 
given by 


n! 
Tk 


Y or ona kann 


Proof. — By adapting the counting procedure in the proof of Theorem 4.5.1 we obtain 
that the function T(x,w) = Yn ko Tn, ew, satisfies 


T (x,w) =x (ero) -1 +w) ; 


Consequently 


It is also of interest to consider the degree of the root. 


Theorem 4.5.3 The number L”) of rooted labeled trees of size n, where the root has 
degree k, is given by 
n—2 = 
EO = i 1) Riga 


In particular, the number of rooted labeled trees, of size n, where the vertex with a 
specified label (e.g. label n) has degree k is given by 


Cy) (n—1yr-1, 


Proof. With the help of the generating function T(x) from the proof of The- 
orem 4.5.1 and by the Lagrange inversion formula (Theorem 4.3.4 applied with 
g(x) =x*) we have 


T(x)f _ nt 1 
i = nifyt eT OP = etre k ae = [a2] Veln— 1) 
In- ss = 


It is also possible to consider sets of trees (or forests). 
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Theorem 4.5.4 The number F;, ; of vertex labeled forests with k rooted tree compo- 


nents is given by 
n =f 


whereas the number Fike of vertex labeled forests with k unrooted tree components is 


given by 
f= (7 )n a S- yri(Tarae as 


Proof. | The generating function for forests that consist of k rooted trees is given by 
T (x)*/k!. Thus we can proceed as in the proof of Theorem 4.5.3. 
Similarly the generating function for forests of unrooted trees is given by 


Again we can apply the Lagrange inversion formula and obtain an explicit formula 
for Fak (after a few lines of computation). | 


Trees are bipartite graphs, that is, we can partition the vertices of a tree into 
two sets V; and V2 so that there are only edges between vertices of V; and V2. For 
example if we fix a vertex vo in a tree then we can define V; as the set of vertices 
with even distance to vo and V2 as the set of vertices with odd distance to vo. Actually 
the partition {V;,V2} is uniquely defined. It is therefore possible to consider labeled 
bipartite trees Dis where V; and V2 have given sizes m and n, respectively, and 
where we label the vertices of V; by 1,...,7 and those of V2 by 1,...,n 


Theorem 4.5.5 The number Tn = | Facal of labeled bipartite trees of sizem-+n 
with labels 1,...,mand 1,...,n, respectively, is given by 


see = myn! . 


Proof. In order to simplify the presentation of the proof we use properly vertex 
colored trees with the colors black and white, where properly colored means that 
adjacent vertices have different colors. (Of course, the black vertices can be consid- 
ered as the set V; and the white ones as the set V2.) Furthermore we label the black 
and white vertices independently. Now let .% denote the set of these trees, where we 
root at a black vertex and .% the set of trees, where we root at a white vertex. The 
corresponding generating functions are 


y" 


= yy, ie . and Th(x,y) = y Bigg i 


m,n>0 'n! m,n>0 


where Tj, j = 1,2, is the number of trees in .7; with m black vertices and n white 
vertices. 
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Similarly to the usual labeled trees there is a corresponding recursive description 
for bipartite trees that leads to the following relations for T; (x,y) and Tp(x,y): 


Ti (x,y) — xe?) and To(x,y) = yell (xy). 


Now Lagrange inversion applied to T; (x,y) =xexp (ver Go) yields: 


= y"] iy ys 


k>01>0 
_ m 1 y m ye ml = mnt! 
im Ak! (m1)! min! 


Furthermore, Tig = Ti nn /m= Tamn /n since there are exactly m ways to choose 
the root of type | in an unrooted tree with m vertices of type 1. | 


Finally we also consider labeled trees together with their embedding into the 
plane. Interestingly there is also an explicit formula for the number of different planar 
embeddings. 


Theorem 4.5.6 The number T,, of different planar embeddings of rooted labeled 
trees of size n is given by 


sd (2n — 3)! 


T,=1 and T,=n (n > 2). 


(n—1)! 7 
Consequently the number of different planar embeddings of unrooted labeled trees 
of size n equals (2n —3)!/(n—1)! (forn > 2). 


Proof. Let P(x) denote the exponential generating function of labeled rooted plane 
trees. Then due to the recursive structure of these kinds of trees we have (compare 
with the proof of Theorem 4.4.3) 


lee 1— P(x) 


Consequently, the exponential generating function T (x) for the numbers 7,, of differ- 
ent embeddings for labeled rooted trees Y = 0+0*Cyc(“) is given by 


, 1 
T(x) =x+xlog —_,—. 
aaa 2 er 


Hence, by Lagrange’s inversion formula (Theorem 4.3.5) we obtain (for n > 2) 


Rn 1 
Th — ne NAG) 
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1 1 1 


—ny! n—2 
irc a? (l—u)r! 


4.5.2 Recursive trees 


Recursive trees are rooted labeled trees, where the root is labeled by | and the labels 
of all successors of any node v are larger than the label of v (see Figure 4.14). 


Figure 4.14 
Recursive tree. 


Usually one does not take care of the possible embeddings of a recursive tree 
into the plane. In this sense recursive trees can be seen as the result of the following 
evolution process. Suppose that the process starts with a node carrying the label 1. 
This node will be the root of the tree. Then attach a node with label 2 to the root. The 
next step is to attach a node with label 3. However, there are two possibilities: either 
to attach it to the root or to the node with label 2. Similarly one proceeds further. 
After having attached the nodes with labels 1,2,...,k, attach the node with label 
k +1 to one of the existing nodes. 

This procedure is actually the basis of the proof of the following enumeration 
result. 


Theorem 4.5.7 The number Ry, of recursive trees of size n is given by 
R, = (n—1)! 


Proof. — Every recursive tree of size n is obtained by the above evolution process 
in a unique way. (Actually the labels represent the history of the evolution process.) 
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Since there are exactly k ways to attach the node with label k + 1, there are exactly 
1-2---(n—1) = (n—1)! possible trees of size n. | 


Note that the left-to-right-order of the successors of the nodes in a recursive tree 
was not relevant in the above counting procedure. It is, however, relatively easy to 
consider all possible embeddings as plane rooted trees. These kind of trees are usu- 
ally called plane oriented recursive trees (PORTs). 


Figure 4.15 
Two different plane oriented trees. 


Theorem 4.5.8 The number P, of plane oriented recursive trees of size n is given by 
P, = (2n—3)!!=1-3-...-(2n—3). 


Proof. _ Plane oriented recursive trees can again be seen as the result of an evolution 
process, where the left-to-right-order of the successors is taken into account. More 
precisely, if a node v has out-degree d, then there are d + | possible ways to attach a 
new node to v. Furthermore we have for a rooted tree 


Ye @) +1) =2|V(7)|-1. 


veV(T) 
Hence, the number of different plane oriented recursive trees with n nodes equals 


1 (2(n—1))! 
2-1 (n—1)! * 


1-3+...+(2n—3) = (2n—3)!! = 
i 


The model of simply generated trees was to define a weight for a tree that reflects 
the degree distribution of rooted trees. The same idea can be applied to recursive and 
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to plane oriented recursive trees. The resulting classes of trees are called increasing 
trees. They have been first introduced by Bergeron, Flajolet, and Salvy [2]. 

As for simply generated grees we define the weight @(T) of a recursive or a plane 
oriented recursive tree T by 


o(T)= [I ga+) =T1 9)”, 


veV(T) 720 


where d*(v) denotes the out-degree of the vertex v (or the number of successors) and 
Dj(T) the number of nodes in T with j successors. Then we set 


Y=}, a7); 
TE fn 
where _Y,, denotes the set of recursive or plane oriented recursive trees of size n. 
We again introduce the generating series 
x e 
P(x) = Pot Pix + Pry, + Paap to 


in the case of recursive trees and 
D(x) = Pot Gix+ Gox* + 93x +++ 


in the case of plane oriented recursive trees. The generating function 


r@= Py 


n>0 
satisfies the differential equation 
Y'(x) =®(¥(x)), ¥(0)=0. 


In the interest of clarity we state how the general concept specializes. 

Recursive trees (that is, every non-planar recursive tree gets weight 1) are given 
by ®(x) = e*. Here Y, = Ry, = (n— 1)! and R(x) = log(1/(1—x)). 

Plane oriented recursive trees are given by B(x) = 1/(1 —x). This means that 
every planar recursive tree gets weight 1. Here Y,, = P, = (2n—3)!! =1-3-5---(2n— 
3) and Y(x) =1—V1—2x. 

Binary recursive trees are defined by @(x) = (1 +.x)*. We have Y, =n! and 
Y (x) = 1/(1—x). The model that is induced by this (planar) binary increasing trees 
is exactly the standard permutation model of binary search trees that is discussed 
in Section 4.6.3. 


4.5.3. Well-labeled trees 


Well-labeled trees are planted plane trees, where the vertices v are labeled with inte- 
ger numbers ¢(v) > 1 such that the root gets label £(root) = 1 and labels ¢(v), é(w) 
of adjacent vertices satisfy 

|é(v) — &w)| <1. 


See Figure 4.16 for an example. 
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Figure 4.16 
Well-labeled tree. 


Well-labeled trees (and variations of them) have been discussed quite intensively 
during the last few years. This comes from the fact that there is a remarkable bijection 
between (rooted) quadrangulations with n faces and well-labeled trees with n edges 
that is due to Gilles Schaeffer [24]. One of the main features of this bijection is 
that the labels of the well-labeled tree correspond to the graph distance to the root 
of the (corresponding) quadrangulations. This observation made it possible to study 
the distance profile of quadrangulations and to prove that the diameter of a randomly 
chosen quadrangulation is of order n'/4, see [8]. 

At first sight it is by no means obvious how these kinds of trees can be counted. 
Of course, the resulting number has to coincide with the number of quadrangulations. 
Nevertheless there is a direct proof that uses an ingenious method due to Bouttier, Di 
Francesco, and Guitter [5, 6]; see also Bousquet-Mélou [4]. 


Theorem 4.5.9 The number W,, of well-labeled trees (with increments 0 and +1) 
with n edges, is given by the formula 


2.3" 2n 
= oon): cay 


Proof. | We consider variants of well-labeled trees, where the root has a label not 
necessarily equal to | but again all vertices are labeled by positive integers and labels 
of adjacent vertices differ at most by 1. For j > 1, let Wj(x) denote the (ordinary) 
generating function of those generalized well-balanced trees where the root has la- 
bel j and where the exponent of x counts the number of edges. Then, by using the 
convention Wo(x) = 0, we immediately get the relation 


1 


Ge 1 —x(Wj—1 (x) +Wj(x) + Wj41(x))’ 


(j>1). (4.16) 
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This relation comes from the usual decomposition of a planted plane tree into the 
root and its subtrees. If the root has label j then the successors of the root must 
have a label in the set {j — 1, 7, 7-+1} and the subtrees are again well-balanced trees. 
This justifies (4.16). Note also that the infinite system (4.16) uniquely determines 
the functions W;(x). In particular, the equations (4.16) provide a recurrence for the 
coefficients [x”] W; (x) that can be solved by using the initial condition Wo(x) = 0. For 
example we have indeed [x°] Wo(x) = 0 and [x°] W;(x) = 1 for j > 1. Hence, it follows 
that [x!] Wi (x) = 2 and [x!]W;(x) =3 for j > 2. In the same way we can proceed 
further. If we already know [x*] W;(x) for all k <n and all j > 0 then [x”*"] W;(x) can 
be computed for all j > 1. Since [x”*!] Wo(x) = 0 the induction works. 
Let W(x) = (1 — V1 — 12x)/(6x) be the solution of the equation 


1 
WO Tawa) (4.17) 
and let Z(x) be defined by 
Z(x)4 a l= : 4.18 
Oa)! Ge ae 


Note that one of the two possible choices of Z(x) represents a power series in x with 
constant term 0 (and non-negative coefficients) which we will use in the sequel. 


By using the ansatz 
UU j13 
Wj (x) =W(x inet hat 
, Uj+1Uj+2 


with unknown functions u;, the recurrence (4.16) is equivalent to 
UjUj+{Uj+2QUj43 = Paps (4.19) 
; : ; W(x) J J 
+xW (x) (uj 10420543 + UGG 3 + Ujuj yu ja) - 
By using (4.17) and (4.18) it is easy to check that 
uj=1-Z (x)! 
satisfies (4.19). Hence, the functions 


(1-Z(x)/)(1-Z(x)/**) 


(1-Z@)F1)(1-Z(x)*) (4.20) 


Wj (x) = W(x) 


satisfy the system of equations (4.16) and we also have Wo(x) = 0. Since all these 
functions W;(x) are power series in x, we have found the solution of interest of the 
system (4.16). 

In particular, we have 


(1-Z(x))(1-Z(a)*) 


Wi (x) = W(x) (1 —Z(x)?)(1 —Z(x)3) 
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es +Z(x) +Z(x)? 

= W(x)(1—xW(x)’), 


and it is an easy exercise (by using Lagrange’s inversion formula) to show that 


: 7 2.3" 2n 
Qn={x mie = oe (7), 


This completes the proof of the theorem. | 


The unexpected part of the preceding proof is formula (4.20). The resulting For- 
mula (4.15) for W,, has, however, a quite natural interpretation using the bijection 
between quadrangulations with n faces and well-labeled trees with n edges (the Scha- 
effer bijection [24]). 

There are many more results on well-labeled trees (see [4]). We just mention that 
there is, for example, a similar explicit formula for the number of well-labeled trees, 
where adjacent vertices v, w satisfy |¢(v) — &(w)| = 1. Here we have 


= ean) 


for these kinds of trees with n edges. 


4.6 Selected topics on trees 
4.6.1 Spanning trees 


Let G be a connected graph (without loops or multiple edges). Then a tree T is a 
spanning tree of G if it has the same vertex set as G and if the edge set of T is a 
subset of the edge set of G. Alternatively T is the maximal subgraph of G that is a 
tree. There always exists a spanning tree. We just have to delete edges that are within 
a cycle till we are left with a tree. 

The enumeration problem for spanning trees is solved by Kirchhoff’s matrix tree 
theorem (see for example [21]). 


Theorem 4.6.1 Let G be a connected graph without multiple edges and loops. Let 
{v1,v2,-.-;Vn} denote its vertex set, d(v1),d(v2),...,d(vn) the degree sequence and 
D = diag(d(v1),d(v2),...,d(vn)) the corresponding diagonal matrix. Furthermore 
letA = (Gi i<iz<n denote the adjacency matrix of A, that is, aj; = 1 if v; and v; are 
linked by an edge and aj; = 0 else. 

Then the number of spanning subgraphs of G equals the absolute value of the 
determinant of any (n—1) x (n—1) submatrix of D—A. 
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Figure 4.17 
An illustration of the use of the matrix tree theorem. 


Actually much more can be said. For example, let us consider the graph shown 
in Figure 4.17 with four vertices, where we also label the edges by e1,...,e5. 

We then consider a generalized version D of the diagonal matrix D, where the 
degrees are replaced by the sum of the edges that are incident with the corresponding 
vertices. Similarly we define A as a refined adjacency matrix, where a; j 1s replaced 
by the actual edge that connects v; and v; (if it exists): 


ey teq+es 0) 0 0 QO e; e5 4 

p= 0 ey +e2 0 0 yo ey O e O 
0 0) e2 +€3 +65 0 : ee e O @63 

0 0 0 e3+e4 e 0 ee O 


Of course, if we set e; = 1, 1 < j <5, then we obtain the original matrices D and A. 
If we now delete the first row and column of D— A and compute the determinant, we 
get 


ey +e2 —e2 0 
—e2 2 +e34+e5 = —e3 
0) —e3 e3+e4 


= €1€7€3 + €1€2€4 + €1€3€4 + €1€3€5 + €1 €4€5 + €7€3€4 + €2€3€5 + €7€465. 


By inspecting this sum we observe that each of these eight monomials corresponds 
to precisely one of the eight possible spanning trees, where the edges are just the 
factors of the monomials. This means that this determinant is the generating series 
of all spanning trees that can be used to do a more refined analysis of spanning trees 
(for example we can count how many spanning trees use the edge e, etc.). 

As mentioned in Section 4.5.1 a labeled tree can be considered as a spanning 
tree of the complete graph K,,. By applying Theorem 4.6.1 we are led to compute the 
(n—1) x (n— 1)-determinant 


a ere 
et ee | 


t- Si es) ad 
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Now we do the following row computations. First we add all but the first row to the 
first row so that the first row consists just of 1’s. Then we add this new first row to all 
other rows and obtain the following determinant: 


1 1 1 11 1 
af eo a4 On 0 
dy = a 
a) «a n-1 0 0 n 


Now it is clear that d, = n"~? 


elements. 

There are several other applications of this method and also an extension to di- 
rected graphs (for more details see Moon’s book [21]). 

Most of the proofs of Theorem 4.6.1 are quite involved and use multi linear alge- 
bra (see Chapter 1 or [21]). However, there is a nice recent analytic proof by Amitai 
Zernik [25] that we can present here. 


since we just have to take the product of the diagonal 


Proof. Let L = (L;;) =A —D. Then L is symmetric and has the property that for 
all i the sums )' ,L;; = 0. Let C;; denote the cofactor of L corresponding to row i and 
column j, that is, (—1)'*/detM;;(L), where Mj;(L) denotes the (n—1) x (n—1)- 
matrix that is obtained from L by deleting the ith row and the jth column. It is easy 
to see that in our case (where )’ ,L;; = 0 for all i) all cofactors have a common value 
that we denote by C(L). 

For every tree T with vertex set {1,2,...,n} we set 


Ar(L)= [J] Ly. 
i<j, {i,j}€E(T) 


The main observation is that 
C(L) =) Ar(L). 
T 


This is precisely the generalized version of Theorem 4.6.1 (note that the diagonal is 
not used to compute Ar (L)), which specializes to Theorem 4.6.1 if we set L=A—D. 

We denote by the linear space of all symmetric matrices L = (Lj) with the 
property that for all i the sums )' | Lj; = 0. It is easy to see that Y is path connected 
and that a (continuously differentiable) function f on & is constant if and only if 
D;;f =0 for alli < j, where Dj; is defined by 


0 0 7] 0 


Di = , 
TOL OL OL OL jj 


(Formally Dj;, i < j, span the tangent space of & at every point of ..) Next set 
B(L) =¥.7Ar(L). The idea of the proof is to compare the derivatives D;; of both sides 
of the proposed equality C(L) = B(L). More precisely we show that D,;(C(L)) = 
Dj;(B(L)) for all i < j. Since C(0) = B(O) = 0 the equality C(L) = B(L) follows for 
allLe &. 
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Let L’ = M;;(L1.), where L is an (n—1) x (n— 1)-matrix obtained from L by 
adding the jth row to the ith row and the jth column to the ith column. Note that L’ 
has the same properties as L, that is, it is symmetric and )’ Li, , = 0 for all i’. The 
main step of the proof is to show that D;;(B(L)) = B(L’) and Dj;(C(L)) = —C(L’). 
The result then follows by induction on the size of the matrix. 

To see that D;;(B(L)) = B(L’), note that D;;(Ar(L)) = 347 (L) is nonzero only 
if i, 7 are connected by an edge in T, in which case we may contract the edge to pro- 
duce a new spanning tree T’ = contract; ;(T) with vertices {1,2,...,n—1}. Explicitly, 
we erase the vertex j and the edge {i,j} and reconnect all the other neighbors of j 
to the vertex i. We also relabel the vertices j+1,...,1 by j,...,n— 1, respectively. 
Fixing some 7" it is then easy to see that 


ie. 
aL 


T:contract;;(T)=T' ij 


Ar(L) =Ar'(L’) 


which implies D;;(B(L)) = B(L’). 
Finally we show —D,;(C(L)) = C(L’). Indeed, 
0 0 


OL jj OL ji 


—Di;(C(L)) = —Dij(Gi(L)) = Cyi (Ly 


a 


=— 52 det(Mi(L4)) = det(Mj—1i-1(Mii(L+))) — det Mji(Mj;(L+)) 


= detM;(L') =C(L’), 


which completes the proof. | 


4.6.2 k-Trees 


A k-tree is a graph that can be considered as a generalized tree and can be defined 
recursively: A k-tree is either a complete graph on k vertices or a graph obtained from 
a smaller k-tree by adjoining a new vertex together with k edges connecting it to a 
k-clique of the smaller k-tree and, thus, producing a (k + 1)-clique. (A k-clique is a 
subgraph that is isomorphic to a complete graph on k vertices.) In particular, a 1-tree 
is a usual tree. Figure 4.18 shows an example of a 2-tree. 

The notion of a k-tree originates from the parameter tree-width tw(G) of a graph 
G, which is the minimum width among all possible tree decompositions of G, or 
equivalently, tw(G) is the minimum k such that G is a subgraph of a k-tree. A k-tree, 
as a bounded tree-width graph, is exactly the maximal graph with a fixed tree-width 
k such that no more edges can be added without increasing its tree width. 

Labeled k-trees have been already counted by Beineke, Pippert, Moon and Foata 
[1, 21, 14] four decades ago. 


Theorem 4.6.2 Let k > 1. Then the number Bw of k-trees having n labeled vertices 
is given by 


Bw = (1) (k(n—k) $1)". (4.21) 
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Figure 4.18 
A k-tree. 


We present here a proof that is related to generating functions (see [9]). 


Proof. We call a k-clique of a k-tree front and a (k + 1)-clique hedron. It is clear 
that k-tree with n vertices contains precisely n — k hedra. In the following proof we 
will work with k-trees with n hedra (and thus with n+ k vertices). 

First we introduce a bijection between labeled k-trees and so-called k-front cod- 
ing trees. A k-front coding tree has unlabeled black nodes and labeled white nodes. 
To construct a k-front coding tree from a labeled k-tree, an unlabeled black node in 
a k-front coding tree represents a hedron in the labeled k-tree. A white node labeled 
by the set {i,i2,...,i,} represents a front with vertices that are labeled by the in- 
tegers i1,i2,...,i, 1n the labeled k-tree. A black node connects with a white node if 
the corresponding hedron contains the corresponding front. It is clear that labeled 
k-trees having n hedra are in bijection with k-front coding trees with 1 black nodes 
and kn + 1 labeled white nodes. See Figure 4.19 for an example. 


1 4 QD 


Figure 4.19 
Bijection between a labeled 2-tree (left) and a 2-front coding tree (right). 
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We will next reduce the problem of counting unrooted trees into rooted trees. 
This can be done by rooting a k-front coding tree at one of its fronts. Suppose a k- 
front coding tree is rooted at a white node labeled by the set {i1,i2,...,i,}. Then we 
label the black nodes in this k-front coding tree as follows. For every hedron whose 
vertices are labeled by the integers j, j2,.-.-, /41 ina k-tree, the corresponding black 
node connects to white nodes labeled by the k-subsets of {/1, j2,.--, jx41} in the k- 
front coding tree. We will label this black node by an integer j,, if among all its 
neighbors, the white node labeled by the set {j1, j2,---,jx+1}\ {jm} is closest to the 
root. In this way we get a k-front coding tree rooted at {i1,i2,...,ig} with all black 
nodes labeled. So a labeled k-tree rooted at a front can be identified with a k-front 
coding tree rooted at a white node. See Figure 4.20 for an example. 


Figure 4.20 
Bijection between a labeled 2-tree rooted at a front whose vertices are labeled by 1,2 
(left) and a 2-front coding tree rooted at a white node labeled by {1,2} (right). 


A jm-teduced black-rooted tree is a k-front coding tree rooted at a black node 
with labeling j,, and if this black node represents the hedron whose vertices are 
labeled by ji, j2,---, 7x41, then all the neighbors of this root are labeled by the k- 
subsets of {j1, j2,---, j¢+1} that contain j,,. A permutation of the set {x1,x2,...,x,} 
is a bijection 7 : {x1,x2,...,x,} > {x1,%2,-..,x~} such that we write 1 = i; i2---ix 
if 2(xm) = im. We further observe every j,,-reduced black-rooted tree is fixed by any 
permutation of the set {j1, j2,---,je+1}\ {jm} and every k-front coding tree rooted 
at a white node labeled by the set {i),i2,...,i,} is fixed by any permutation of the set 
{i,,i2,...,ig}. Let RY, denote the number of labeled k-trees having n hedra that are 
rooted at a particular front whose vertices are labeled by i), i2,...,i. This number 
also counts k-front coding trees on 7 labeled black nodes and kn + | labeled white 
nodes, which are rooted at a white node labeled by the set {i1,i2,...,i,}. The number 
R,(n+k) is independent of the choice of the particular root {i1,i2,...,ig}. Let Cy (x) 
be the exponential generating function for {i),i2,...,i,}-rooted k-front coding trees 
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where the size is the number of black nodes, that is, 

eS xn 

C(x) = Yo Ry(n +k) 

n=0 . 


Let B,(x) be the exponential generating function for j,,-reduced black-rooted k- 
front coding trees. Since every k-front coding tree rooted at a white node labeled 


by {i1,i2,...,ix} can be identified as a set of reduced black-rooted trees that are fixed 
by any permutation of {i,,i2,...,i,}. Thus we have 
Ci (x) = exp(Bx(x)). (4.22) 


We observe that the black root j,, in a reduced black-rooted tree connects with k 
white-rooted coding trees. All these k white-rooted coding trees are respectively 
rooted at the white nodes labeled by the k-subsets of {i1,i2,...,ix, jm} except 
{i1,i2,...,ix}. This yields 


Bx (x) = x(C(x))4 
and combining it with (4.22) 
By (x) = xexp(kBx(x)). (4.23) 


By applying Lagrange’s inversion formula to (4.23), we obtain 


E"Be(x) = tr Heme — Ged 


which implies that the number of j,,-reduced black rooted k-front coding trees having 
n black nodes is (kn)"~!. Similarly we can derive the coefficients of C,(x) from 
equation (4.22): 


n 1 n— nN u kn + 1 a 
"IC Cx) = 2 ur Helin = rt 1 


(k) 


Equivalently, R (kn+1)"~! counts the number of labeled k-trees having n hedra 


n+k 
that are rooted at a particular front whose vertices are labeled by 71, i2,...,i,%. Since 
there are () ways to choose the set {i1,i2,...,ix}, the number of labeled k-trees 


having n hedra that are rooted at a front is 
(&) _ (2 +K) pw 
(kn aa Bik = ( k je 


and the explicit formula for Bw given in (4.21) follows. | 


The counting problem of unlabeled k-trees is much more difficult. Only the case 
of 2-trees was already solved by Harary and Palmer [18] and Fowler et al [15] by 
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using the dissimilarity theorem. The general case was a long-standing open problem 
and was solved just recently by Gainer-Dewar [16]; see also [17]. 

In order to state the result (as given in [17]) we have to introduce some notation. 
Let g € G,, be a permutation of {1,2,--- ,m} that has @; cycles of size i, 1 <i <k, in 
its cyclic decomposition. Then its cycle type A = (1/1 2” .-. k“«) is a partition of m 
where m = ¢; +20) +---+k0,. We will denote by A | m that A is a partition of m. 
Furthermore we set z, = 110,122 65!.--k%&b,! and 2 is the number of permutations 
in G» of cycle type A. 

The result says that the generating function U(x) = Yys, Un of unlabeled k- 
trees is given by 

U(x) = B(x) + C(x) — E(x), 


where 
B(x) = ae Bails) = x] Tuto) 
AFk+1 
Cu(2) Bulx) = xT [Cu(*') 


CE) =. 5 ae i 
rk 7 Lg co Bym(x") 
EG) = y “Heke > | 


Q 
= 
— 
S 
wa 
i?) 
Pal 
uo) 


urk Su 

Here A is always the cycle type of a permutation in G;1; and p the cycle type of a 
permutation in G,. Furthermore A! denotes the cycle type of permutation z' where 
m € Gg+1 has cycle type A and i is a part of A. The above products over i range 
over all parts i of A or 1, respectively, that is, if i is contained m times in A then it 
appears m times in the product. Finally we implicitly use the relation By (x) = By (x), 
Cy (x) = Cy (x) if A is obtained from ys by adding a part 1. So starting from fp = i 
where we get an implicit equation for B)x«(x), we can compute step by step all in- 
volved generating functions. 


Recently the asymptotic behavior of u,® was determined in [10]: 


uw ~ cryin !?, 


where cx, and % are certain positive constants. 

Similar to recursive (and increasing) trees we can consider all possible evolution 
processes of k-trees, when we start with a specified complete graph (on k vertices) 
and add step by step a vertex and its corresponding edges. This means that we have 
k unlabeled vertices from the starting complete graph and n labeled vertices that 
encode the evolution. In each step we have ik + 1 choices for the new vertex with 
label i+ 1. Thus the resulting number of increasing k-trees (with k +n vertices) is 


RW =] (ik +1). 
i=0 


The corresponding (exponential) generating function R(x) = L,50 RY x /n! satisfies 
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the differential equation 


and is explicitly given by R(x) = (1 —kx)7!/4. 


4.6.3 Search trees 


Search trees are used in computer science for storing and searching data. We describe 
here a few standard models, where the underlying tree is binary. The counting prob- 
lem is not the main focus here, since this has been already done in Section 4.4.1. 
However, the combinatorial and algorithmic background are of considerable interest. 
Furthermore these kinds of trees give rise to very interesting probabilistic tree mod- 
els, however, we will not go into these details. We refer to the monographs [11, 19]. 

We start with binary search trees. The origin of binary search trees dates to a 
fundamental problem in computer science: the dictionary problem. In this problem 
a set of records is given where each can be addressed by a key and the purpose is 
to maintain this set under insertion, deletion and membership queries. The binary 
search tree is a data structure used for storing the records in its internal nodes (in a 
way that will explained below). It will then also be clear how the operations insert, 
delete and search can be implemented then. 

In principle binary search trees are plane binary trees generated by a per- 
mutation (or list) a(1),a(2),...,a(n) of the numbers {1,2,...,2}. The elements 
m(1),2(2),...,@(n) serve as keys and will be stored in the internal nodes of the 
resulting tree. Equivalently we could think of a list of keys, where each pair of keys 
can be compared. Of course, this is the situation in practice. 

Starting with one of the keys (for example with the first on the list 7(1)) one first 
compares (1) with (2). If 7(2) < 2(1), then 2(2) becomes root of the left subtree; 
otherwise, 7(2) becomes root of the right subtree. When having constructed a tree 
with nodes z(1),...,7(k), the next node 2(k + 1) is inserted by comparison with the 
existing nodes in the following way: Start with the root as current node. If 7(k + 1) is 
less than the current node, then descend into the left subtree, otherwise into the right 
subtree. Now continue with the root of the chosen subtree as current, according to 
the same rule. Finally, attach n+ 1 external nodes (= leaves) at the possible places. 
Figure 4.21 shows an example of a binary search tree (without and with external 
nodes) for the input keys (4,6,3,5,1,8,2,7). 

Alternatively one can describe the construction of the binary search tree recur- 
sively in the following way. If n > 1, we select (as above) a pivot (for example 2(1)) 
and subdivide the remaining keys into two sublists 1), h: 


Ty = (x € {m(2),...,a(n)}:x<a(1)) and 
Ih = (x € {m(2),...,a(n)} sx > a(1)). 


The pivot (1) is put to the root and by recursively applying the same procedure, the 
elements of J; constitute the left subtree of the root and the elements of J, the right 
subtree. This is precisely the standard Quicksort algorithm. 
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Figure 4.21 
Binary search tree. 


Now the operations insert and search are easy to implement. If we want to insert 
a new record we just compare its key with the key of the root. If it is smaller then we 
go to the left subtree and if it is larger we go to the right subtree. Then we iterate this 
procedure till we find an external node, where we can store this record. The searching 
procedure is almost the same (the only difference is that we stop if the keys are the 
same). Finally if we want to delete a record r, we first have to find it and then we 
have to distinguish between two cases. First if the corresponding internal node i(r) 
has no or just one internal node as a successor we just delete the internal node i(r) 
(and thus r) and contract the tree. Second, if i(r) has two internal nodes as successors 
replace the value at i(r) with the largest value in its left subtree and then delete the 
node with the largest value from its left subtree. (Note that largest value in the left 
subtree will never have two subtrees so this node can be safely deleted as in the first 
case.) 

The standard probabilistic model for binary search trees (and their variants) is 
the so-called permutation model, where it is assumed that every permutation of the 
input data is equally likely. We again refer to the books [11, 19], where several tree 
parameters of interest are studied under this model. 

Digital search trees are intended for the same kind of problems as binary search 
trees, that is to insert, to delete, and to search. However, they are not constructed 
from the total order structure of the keys for the data stored in the internal nodes of 
the tree but from digital representations (or binary sequences) that serve as keys. 

Consider a set of records, numbered from | to n and let x),...,x, be binary se- 
quences for each item (that represent the keys). We construct a binary tree, the digital 
search tree, from such a sequence as follows. First, the root is left empty, we can say 
that it stores the empty word. Then the first item occupies the right or left child of the 
root depending on its first symbol. If the first symbol is 1 then it occupies the right 
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one, otherwise the left one. After having inserted the first k items, we insert item 
k +1: Choose the root as current node and look at the binary key x,+,. If the first 
digit is 1, descend into the right subtree, otherwise into the left one. If the root of the 
subtree is occupied, continue by looking at the next digit of the key. This procedure 
terminates at the first unoccupied node where the (k + 1)th item is stored. 

For example, consider the items 


x; =110011--- x5 =000110--- 
x2 = 100110--- x6 =O10111--- 
x3 =010010--- x7 = 000100: -- 
x4 = 101110--- xg = 100101---. 


If we apply the above described procedure we end up with the binary tree depicted 
in Figure 4.22. As in the case of binary search trees we can append external nodes to 
make it a complete binary tree. 


Figure 4.22 
Digital search tree. 


The standard probabilistic model, the Bernoulli model, is to assume that the keys 
X1,...,X, are binary sequences, where the digits 0 and | are drawn independently and 
identically distributed with probability p for 1 and probability g = 1 — p for 0. The 
case p=q= 5 is called symmetric. 

There are several natural generalizations of this basic model. Instead of a binary 
alphabet one can use an m-ary one leading to m-ary digital search trees. One can also 
change the probabilistic model by using, for example, discrete Markov processes to 
generate the key sequences. 

The construction idea of tries is similar to that of digital search trees except 
that the records are stored in the leaves rather than in the internal nodes. Again a 
1 indicates a descent into the right subtree, and 0 indicates a descent into the left 
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subtree. Insertion causes some rearrangement of the tree, since a leaf becomes an 
internal node. In contrast to binary search trees and digital search trees, the shape 
of the trie is independent of the actual order of insertion. The position of each item 
is determined by the shortest unique prefix of its key. If we use the same input data 
as for the example of a digital search tree, then we obtain the trie that is depicted in 
Figure 4.23. 


Figure 4.23 
Trie. 


An alternative description runs: Given a set 2 of strings, we partition 2 into 
two parts, 27 and 2%, such that x; € 2 (respectively x; € Zp) if the first symbol 
of x; is 0 (respectively 1). The rest of the trie is defined recursively in the same way, 
except that the splitting at the kth level depends on the kth symbol of each string. The 
first time that a branch contains exactly one string, a leaf is placed in the trie at that 
location (denoting the placement of the string into the trie), and no further branching 
takes place from such a portion of the trie. 

This description implies also a recursive definition of tries. As above consider a 
sequence of n binary strings. If n = 0, then the trie is empty. If n = 1, then a single 
(external) node holding this item is allocated. If n > 1, then the trie consists of a 
root (internal) node directing strings to the two subtrees according to the first letter 
of each string, and string directed to the same subtree are themselves tries, however, 
constructed from the second letter on. 

As in the case of digital search trees we can construct m-ary tries by using strings 
over an m-ary alphabet leading to m-ary trees. 
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5.1 Introduction 


The enumerative theory of planar maps is born in the early sixties with the seminal 
work of William T. Tutte on the enumeration of planar triangulations. Over 50 years 
it has led several parallel lives in combinatorics, statistical physics, quantum gravity, 
enumerative topology and probability theory, that have started to interact intensely 
only in the last ten to fifteen years. Writing a fair survey of this whole story appears 
to be a great challenge. We concentrate instead in this text on the combinatorial point 
of view on map enumeration, and only review very tangentially the physics, topology 
and probability literature. 


5.2. What is a map? 
5.2.1 A few definitions 


Combinatorial maps usually arise from one of two settings: either the study of some 
planar graph drawings, or the construction of surfaces via polygon gluings. Accord- 
ingly we give two definitions of maps and discuss how duality reconciles them. 


The graph drawing point of view. An embedding (or drawing) of a graph G = 
(V,E) on the oriented sphere S is proper if the vertices are represented by distinct 
points and the edges are represented by arcs that only intersect at their endpoints and 
in agreements with the incidence relation of G. 


Definition 5.2.1 A planar map M is a proper embedding of a connected graph G in 
the sphere S, considered up to orientation preserving homeomorphisms of S. 


Loops and multiple edges are allowed, and the map is instead said to be simple 
if it contains neither multiple edges nor loops. A face is a connected component of 
S\G. A corner is the angular sector delimited by two consecutive edges around a 
vertex. Each corner c is incident to a vertex v(c), to a face f(c), and to two edges: In 
counterclockwise direction around v(c), let cw(c) denote the edge preceeding c and 
ccw(c) denote the edge following c. The degree of a vertex or face is the number 
of incident corners. A map is Eulerian if all its vertices have even degrees. It is 
m-valent if all its vertices have degree m, it is a m-angulation if all its faces have 
degree m. In the special cases m = 3,4 we use the standard terminology trivalent 
maps and tetravalent maps, triangulations and quadrangulations. Observe that with 
these definitions, triangulations and quadrangulations are allowed to have multiple 
edges or loops. Some examples are given in Figure 5.1. Observe that the leftmost 
map is not simple because the edge e’ has a parallel edge. 
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Figure 5.1 

Three planar maps. The first has 10 vertices, 14 edges and 6 faces. The second is a 
tetravalent triangulation with 8 triangles. The third is a trivalent quadrangulation with 
6 quadrangles; it is usually called the cube. On the first map the corner c is incident 
to the vertex v = v(c), to the face f = f(c), and the edges e = cw(c) and e’ = ccw(c). 


o = (1,2,3,4)(5, 6, 7,8)(9, 10, 11, 12)(13, 14, 15, 16)(17, 18, 19, 20)(21, 22, 23, 24) 
p = (1,10)(2, 8)(3, 24)(4, 14)(5, 9)(6, 20)(7, 21) (11, 13)(12, 17)(15, 23)(16, 18)(19, 22) 


Figure 5.2 
The cube as a cellular decomposition of the sphere. To make the picture more read- 
able, the side identifications are performed in two stages. 


The polygon gluing point of view. A cellular decomposition of the sphere is a 
collection of oriented polygons with labeled corners, and a complete set of coherent 
side identifications such that the resulting surface is the sphere. An example of cel- 
lular decomposition of the sphere is given by Figure 5.2. More precisely, a cellular 
decomposition can be given by the associated rotation system (o,/—), consisting of 
a permutation o whose cycles describe the clockwise arrangement of corner labels 
around polygons and a matching (or fix point free involution) p describing side iden- 
tifications: If p(i) = j then the polygon side (i,o(i)) is identified with the polygon 
side (j,0(j)). 

Definition 5.2.2 A planar map is a cellular decomposition of the sphere considered 


up to relabeling of the corners of the polygons. 


Equivalence between Definitions 5.2.1 and 5.2.2 follows from standard results in 
surface topology: The faces of a proper embedding of a connected graph G in S are 
simply connected, so that the components of S \ G can be identified with polygons, 
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A(M) M* Q(M) R(M) 


Figure 5.3 

The cube map M, and in the first line, the construction of its dual, incidence map and 
edge map. The resulting maps A(M), M*, Q(M) and R(M) appear on the second line. 
The underlying spheres are omitted. 


and conversely, in a cellular decomposition the sides of the original polygons define 
a proper embedding of a connected graph on the resulting surface. In particular the 
numbers v(M) of vertices, f(M) of faces and e(M) of edges of a planar map M satisfy 
Euler’s formula: 

v(M) +f(M) = e(M) +2. (5.1) 


The dual of a map. There is a natural symmetry between the role of vertices in 
Definition 5.2.1 and the role of polygons in Definition 5.2.2. This observation directly 
leads to the fundamental idea of duality, illustrated by Figure 5.3. The dual of a map 
M, denoted M*, is the map obtained by drawing a vertex f* of M* in each face f of 
M and an edge e* of M* across each edge e of M (see Figure 5.3). By construction 
each face of M* then contains exactly one vertex of M. The superimposition of a 
map M and its dual M* (with tetravalent vertices created at the intersection of dual 
edges) is a quadrangulation A(M), which is called the derived map of M. Observe 
that faces of A(M) are in one-to-one correspondence with corners of M. 


Theorem 5.2.3 Duality is an involution on the set of planar maps. It preserves 
the number of edges, and exchanges the numbers of vertices and faces: M** = M, 
e(M*) = e(M), and v(M*) = f(M). 


Let M be a map with vertex set V and edge set E, and let M* be its dual with 
vertex set V* and edge set E*. The incidence map (or quadrangulation) Q(M) of 
the map M is the map whose vertex set is VUV* and with one edge per corner c 
connecting v(c) and f(c)*. The edge map R(M) of M is instead the map with vertex 
set E and with one edge per corner c connecting cw(c) and ccw(c). The mapping 
Q and R are bijections from maps with n edges respectively onto vertex-bicolored 
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Figure 5.4 
The superimposition of a map M and its dual M*, and two dual spanning trees. 


quadrangulations with n faces, and onto face-bicolored tetravalent maps with n ver- 
tices. Moreover Q(M) = Q(M*), R(M) = R(M*) and Q(M) = R(M)*, where the bar 
denotes the exchange of colors. 

The transformation between M, M*, A(M), Q(M) and R(M) should be viewed 
as mere changes of representations for a same underlying object: In the language 
of computer science, one would say that they represent different data structures that 
can be used to encode a same cellular decomposition of the sphere. Alternatively 
this statement can be made precise using the language of topology, and in particular 
branched covers of the sphere (see [125, Chapter 1] for an exposition, which requires 
too many definitions to be reproduced here). 


Duality, spanning trees and Euler’s formula revisited. A spanning tree of a 
planar map M is a subset T of the set of edges of M that forms a tree and that is 
incident to every vertex of M. The picture on the right of Figure 5.4 illustrates the 
following fundamental property of duality and spanning trees for planar maps: 


Theorem 5.2.4 Let (T,7>) be a partition of the edges of a planar map M. Then T, 
is a spanning tree of M if and only if T; is a spanning tree of M*. 


The trees 7; and 7," are usually called dual spanning trees, although this terminol- 
ogy is somewhat improper since the edges of 7, are not the duals of the edges of 
T;, but rather the duals of the edges not in 7,. The proposition can be viewed as a 
consequence of the characterization of the sphere by Jordan’s lemma (S \ 7; is con- 
nected if and only if there is no simple cycle in 7,), together with the fact that S\ T 
is connected if and only if 7," is connected. 

Observe that the above proposition, together with the facts that every map admits 
a spanning tree, and that any tree with v vertices has v — | edges, gives an interpre- 
tation of Euler’s formula (5.1): The e(M) edges of a map are partitioned into the 
v(M) — | edges of a spanning tree and the f(M) — 1 edges of the dual spanning tree. 


5.2.2 Plane maps, rooted maps and orientations 


Rather than drawing maps on the sphere, we usually draw maps on the plane. This 
naturally leads to the notion of rooted maps, and to the discussion of orientations. 
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Figure 5.5 
Three roots for a planar map: the corresponding plane maps, and the plane embed- 
ding of the derived map of the first and third one. 


Plane maps, rooted planar maps. In order to represent a planar map M in the 
plane, we choose a point xo of S in a face of M and identify the punctured sphere 
S? \ {xo} with the plane, sending xo at infinity. In such a representation, all faces are 
homeomorphic to discs, except for the face containing x9, which is usually called 
the exterior or outer face. Depending on the choice of x9 we a priori get different 
drawings, but up to homeomorphisms of the plane only the choice of the face in 
which xo is taken matters. Accordingly, let a plane map (M,/) be a planar map 
M with a marked face f, so that plane maps are in one-to-one correspondence with 
equivalence classes of proper embeddings of connected graphs in the plane up to 
homeomorphisms of the oriented plane. 

Given a planar map M, the choice of a marked face does however not fix in 
general the embedding of the dual map M* and of the derived map AM up to home- 
omorphisms of the plane. As illustrated by Figure 5.5, what is needed for this is the 
choice of a face of AM, or equivalently of a corner of M. Accordingly, let a rooted 
planar map (M,c) be a planar map with a marked corner c. The root face, root 
vertex and root edge of (M,c) are then defined to be respectively f(c) and v(c) and 
ccw(c). (In the literature, a rooted map is sometimes defined as a map with a marked 
and oriented edge, or with a marked half-edge. This definition is equivalent to ours: 
to each oriented edge @ is associated the unique corner c such that v(c) is the origin 
of @ and ccw(c) =e.) 

Upon setting (M,c)* = (M*,c), duality extends into an involution on rooted pla- 
nar maps. The derived map of a rooted map (M,c) is instead naturally endowed with 
a marked face (the face of A(M) that corresponds to c). The incidence map of a rooted 
map (M,c) is a bicolored quadrangulation with a marked edge (the edge e of Q(M) 
that corresponds to c), or equivalently a rooted quadrangulation (taking as root the 
unique corner c’ incident to the root vertex of M and such that ccw(c’) = e). Similarly 
the edge map of a rooted map can be considered as a rooted tetravalent map. 


Proposition 5.2.5 The incidence map Q and edge map R are one-to-one correspon- 
dences between rooted planar maps with n edges and respectively rooted planar 
quadrangulations with n faces, and rooted tetravalent planar maps with n vertices. 


Planar Maps 341 


> 


Figure 5.6 
An Eulerian orientation, the ¢-orientation induced by the dual spanning trees of Fig- 
ure 5.4, and a s-orientation of a triangulation. 


Orientations. Let (Mc) be a rooted map, and @ an orientation of the edges of M. 
A circuit is a directed cycle of oriented edges, it is simple if it visits each vertex 
at most once. Any simple circuit C divides the sphere into a left component, which 
borders the left- hand side of every edge of C, and a right component, which borders 
the right-hand side of every edge of C. 

Let us say that a simple circuit C in a rooted planar map is clockwise if the root 
corner lies in its left component, counterclockwise otherwise. With the convention 
that the outer face of a rooted plane map is its root face, these notions of clockwise 
and counterclockwise circuits coincide with the usual definitions: A simple circuit 
is clockwise if the unbounded component it defines is on its left-hand side, counter- 
clockwise otherwise. 

A function a: V — N is feasible on the planar map ™ if there exists an orien- 
tation @ of M such that for every vertex v of M, a(v) is the outdegree of v. Such an 
orientation is then called an a-orientation. 

A classical example of feasible function is the half-degree function h(v) = 
deg(v)/2 on an Eulerian map (recall that a map is Eulerian if all its vertices have 
even degree). The h-orientations are precisely Eulerian orientations, that is, orien- 
tations such that the in- and out-degree are equal on each vertex. Another example 
is given by the orientation induced by spanning trees on derived maps: let (M,c) be 
a rooted planar map and for each vertex v of the derived map A(M) let t(v) = 0 if 
v is the root vertex of M or M*, t(v) = 1 if v is a non-root vertex of M or M*, and 
t(v) = 3 if v is a dual edge intersection vertex. Then ¢ is feasible since each pair of 
dual spanning trees on M and M* induces a t-orientation of A(M) by selecting as 
unique out-going edge on each non-root vertex of M or M’ the edge going toward its 
father in the tree it belongs to. Finally a third example is Schnyder’s orientations of 
triangulations [158]: Let T be a plane triangulation and s(v) = 1 if v is incident to 
the outerface, s(v) = 3 otherwise. Then s is feasible and the s-orientations are called 
3-orientations. See Figure 5.6 for an illustration. 

Let @ be a feasible function on a plane map (M, f), and let @ be an q@-orientation 
of M. Observe that simultaneously changing the orientation of all the edges of a 
circuit in (M, @) yields another orientation 0” that is still an @-orientation. Moreover 
two a-orientations @ and 0” of a planar map M always differ on a set of edges that 
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form a collection of Eulerian submaps: since any Eulerian graph admits an Eulerian 
tour (that is, a circuit that visit every edge once), it is possible to go from one @- 
orientation to any other by a sequence of circuit reversals. 

The clockwise or counterclockwise orientation of simple circuits can now be 
used to endow the set of a-orientations of a plane map (M, f) with an even nicer 
structure: Let us say that a circuit reversal is increasing if it consists in returning 
a ccw-circuit into the opposite cw-circuit. We admit the following theorem, which 
gives a first illustration of the rich combinatorial properties enjoyed by planar maps. 


Theorem 5.2.6 (Felsner [100]) Let a be a feasible function on a planar map (M, f). 
Then increasing circuit reversals endow the set of Q-orientation of a plane map 
(M,f) with a lattice structure. In particular there exists a unique minimal a- 
orientation in this lattice, which is the unique Q-orientation without cw-circuit. 


5.2.3. Which maps shall we count? 


Let 4", M" and .@' denote the sets of (unrooted) planar maps, rooted planar maps 
and corner labeled planar maps respectively. Then the following four counting prob- 
lems are the most commonly considered: 


1. Count rooted planar maps with n edges, or compute the ordinary generating 
function (gf) 


M(z)= Yo 2) 31422497 +.0(2°). 
Mea" 


2. Count planar maps with 2n labeled corners, or compute the exponential gener- 
ating function 


e(M) 2 


M(2= ¥ cam = 1425 +9-315 +0(2). 


3. Count unrooted planar maps with n edges with a weight | /|aut(M)| per map 
M with automorphism group aut(M), or compute 


M*(z) = 
Mem" 


=| lid Pi. lili ly)24 Q(73), 
jaut(M)| +H(54a)et(s +7 t+ a4+5)2%4+0(z) 


4. Count unrooted planar maps with n edges, or compute 


M(z= ¥ M4) — 14.224 427 4 O(z7). 
Mem" 


Observe that we classify maps according to their number of edges (or corners, equiv- 
alently): In view of duality this is the most natural size parameter to consider, but we 
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The 1, 2 and 9 rooted planar maps with 0, 1 and 2 edges, and the corresponding 1, 2 
and 4 unrooted planar maps. 


shall see that most results can later be refined to take into account the numbers of 
vertices and faces. 

The first three problems are in fact essentially equivalent: On the one hand, rooted 
maps have no nontrivial automorphisms (see below), so that each rooted map admits 
(2e(M) — 1)! distinct labelings such that the root corner has label 1. On the other 


hand, each unrooted map corresponds by definition of aut(M) to ae labeled 
maps. Therefore: 
22d. ¢ 22d 
M'(z) = —M'*(z) = —M*(z). 
@={7M@=7 Me 


Automorphisms of maps. Let (M,r) be a rooted map and assume once again 
that its corners are labeled with {1,...,2n}, with (o,p) the corresponding rota- 
tion system. By definition of rotation systems, two comers are neighboring in M 
if their labels can be mapped one onto the other by o or p: as a consequence, 
the connexity of M implies that for any two corners (c,c’) there exists a sequence 
(T1,..-,T%) of elements of {o,p} such that the labels i and i’ of c and c’ satisfy 
i = 1, 0%0---0%/(i). By definition an automorphism of the map M is a relabeling 
m:{1,...,2n}— {1,...,2n} of its corners that preserves the map M, or equivalently, 
that commutes with o and p: oo” =700, and poa=Z op. An automorphism 
m of the rooted map (M,c) is an automorphism of M that also preserves the la- 
bel ig of the root corner c: 2(i9) = ip. Now according to the previous discussion, 
(i) = 107 0---0 %(ig) for some sequence (T,...,T) in {o,p}*. Using the com- 
mutation relations between 7 and o or p, and the equality 2(i9) = io, this shows that 
m(i) =i: The only automorphism of a rooted map is the trivial one. An important 
consequence of this discussion is that the number of different labelings of the cor- 
ners of a rooted map with 27 edges is always (2n)!, and the number of such labelings 
where the root corner has label | is (2n — 1)!. A more detailed discussion of map 
automorphisms can be found in [88], or [143]. 

As illustrated by Figure 5.7 the fourth problem is not equivalent to the other three. 
Observe that we could have considered a fifth problem, namely to count plane maps 
(i.e. unrooted planar maps with a marked face). This problem is less advertised in 
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the literature but it arises as an intermediary step while counting classes of unrooted 
maps. The literature overwhelmingly concentrates on the rooted (or labeled) case: It 
is technically simpler, and most of the results for rooted maps can a posteriori be 
transfered to unrooted maps. 


5.3 Counting tree-rooted maps 


We start this exposition with rooted planar maps with a marked spanning tree, called 
tree-rooted maps: these maps are the main characters of some simple extensions of 
classical bijections between rooted plane trees, balanced parenthesis words and non- 
crossing (aka planar) arch diagrams. There are at least two reasons to discuss at once 
these bijections: First it is a gentle way to get the reader used to manipulating planar 
maps, by building on the more standard combinatorics of plane trees. Second the 
bijections for tree-rooted planar maps are useful tools to simplify the later description 
of bijections for rooted planar maps. This section is largely inspired by the seminal 
work [28]. 


5.3.1 Mullin’s decomposition 


Our first series of results dates back to the work of Tutte and Mullin in the 60’s. 


Walking around a spanning tree. Let a tree-rooted planar map be a rooted pla- 
nar map (M,c) with n edges, endowed with a spanning tree 7;. The counterclock- 
wise walk around (7,c) induces a total order on the 2n corners of M given by the 
order in which these corners are visited by a 2d little ant traveling on the border of 
the tree T; in counterclockwise direction. This process is illustrated by Figure 5.8. 

Each edge is visited twice during the walk and four symbols can be used to record 
the four types of moves of the ant during the walk: e and ¢ for the first and second 
time it goes across an edge, and o and 6 for the first and second time it goes along 
an edge. The counterclockwise contour code is then the word w on the alphabet 
{o,65,e,6} whose ith letter is the type w; of the ith move of the ant. 

The restriction of the ccw contour code w to the letters {0,3} is the standard 
counterclockwise contour code of the rooted plane tree (T\,c) (see Chapter 4 of the 
present book): In particular it is a balanced parenthesis word, that is, |w|. = |w|s 
and for all prefix w’ of w, |w’|. > |w’|s, where |w]|, denotes the number of letters x in 
the word w. The restriction of the ccw contour code w to the letters {e, 6} is instead 
a direct encoding of the arch diagram formed around 7; by the edges not in 7): It is 
in particular again a balanced parenthesis word, |w|. = |w|s and for all prefix w’ of 
w, |W'|e > |w'|s. 

The word w is thus composed of the shuffle of two balanced parenthesis words 
on the alphabet {0,3} and {e,#} respectively. Mullin’s result is essentially that this 
encoding is one-to-one. 
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Figure 5.8 
The eight first corners visited during a ccw walk around a spanning tree, and the 
full induced numbering of corners. The transition from one corner to the next is 
12374 35 7678 295 107) 11 2 12 * 13 = 14 * 15 > 16 
“S17 + 18 “+ 19 “> 20-21 ++ 22 + 23 4 24-+ 25 + 26-27 428 


Figure 5.9 
(i) The contour walk between a spanning tree and its dual. (ii) The associated cubic 
map with a rooted Hamiltonian cycle. 


ee a 


Figure 5.10 
A shuffle of two arc diagrams. 
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Theorem 5.3.1 (Mullin’s encoding [144]) The contour code is a bijection between 
e tree-rooted planar maps (M,c) with n edges, and 


e shuffles of balanced words on {0,5} and {@,#} of length 2n. 


As a consequence, the number of tree-rooted planar maps with n edges derives 
from the number of ways to shuffle two balanced words of respective length i and j 
withi+j=n. 


Corollary 5.3.2. The number of tree-rooted planar maps with n edges is 


2n 
ys (3) CiCj = Cr Cn41 


i+j=n 
i,j20 
where Cy = AC) denotes the nth Catalan number, and the second expression 


follows from the Chu-Vandermonde identity [15] 


Duality and cubic maps with a rooted Hamiltonian cycle. It is worth observing 
that the contour code is compatible with duality. Let (M,c) be a rooted planar map, 
T, be a spanning tree of M and T,* the dual spanning tree of M*. Then, as illustrated 
by Figure 5.9, an ant performing a counterclockwise walk around 7; simultaneously 
performs a clockwise walk around T,*. Using the same symbol as for the ccw contour 
code, one can define the clockwise contour code of the rooted map (M*,c) endowed 
with the tree 7;", and state the following proposition: 


Proposition 5.3.3 The total order induced by the counterclockwise walk around T, 
starting from c in M is identical to the total order induced by the clockwise walk 
around T;’ starting from c in M*. In particular the counterclockwise contour code of 
(M,c) endowed with T, and the clockwise contour code of (M* ,c) endowed with T; 
are mapped one onto the other by the exchanges 0 ++ @, 54> 8. 


From this primal/dual point of view, it is natural to draw the contour walk as a 
curve @ traveling between the spanning tree T; and the dual spanning tree T;" in the 
superimposition of M and M*, as illustrated by Figure 5.9. The intersections of the 
curve @ with edges of M\ T; and M* \ T;‘ create tetravalent vertices in the middle of 
every half-edge of M and M* that is not in J or 7;‘. In the superimposition of M\ 7), 
M*\ T; and @, each of these new vertices is adjacent to three other new vertices (two 
along @ and one along an edge of M\T or M* \ T,‘) and to one vertex of M or M*. 
Upon keeping only the new vertices and the induced map, one obtains a cubic planar 
map endowed with a rooted Hamiltonian cycle @ (that is a rooted cycle that visits 
every vertex exactly once). Again this construction is bijective. Upon straightening 
the rooted cycle @ it yields a direct geometric intepretation of the contour code as the 
shuffle of two arch diagrams above and below @. See Figure 5.10 for an illustration. 


Theorem 5.3.4 (Mullin [144]) There is a bijection between tree-rooted planar maps 
with n edges and cubic planar maps endowed with a rooted Hamiltonian cycle of 
length 2n. 
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Figure 5.11 
(i) The opening of the tree-rooted map of Figure 5.8. (ii) The resulting balanced tree. 
(iii) The first few steps of the closure of the previous balanced tree. 


Corollary 5.3.5 (Tutte [160]) The number of cubic planar maps endowed with a 
rooted Hamiltonian cycle of length 2n is Vis j=n (3") CiCj = CaCn41- 


Breaking the symmetry, balanced trees. The previous constructions were sym- 
metric with respect to duality. Let us now consider an alternative construction that 
breaks this symmetry. The idea is to break during the ccw walk the edges that are 
crossed into pairs of half-edges. More precisely, in view of the previous discussion, 
it is natural to distinguish for each edge the half-edge that corresponds to the first visit 
and the half-edge that corresponds to the second visit: This is done by using outgoing 
half-edges for the first ones, and incoming half-edges for the second ones. The result- 
ing operation, called the opening, turns a tree-rooted map M into a rooted decorated 
plane tree, that is, a rooted plane tree with two types of dangling half-edges, as illus- 
trated by Figure 5.11. In view of the previous discussion, the decorated trees that are 
produced in this way are characterized by the fact that the number of outgoing and 
incoming half-edges are equal, and that the sequence of outgoing and incoming half- 
edges that are met during a counterclockwise walk around the tree forms a balanced 
parenthesis word. These rooted decorated trees are called balanced trees. 

Conversely the fact that the outgoing and incoming half-edges of a balanced tree 
form by definition a balanced parenthesis word ensures that there is a unique way 
to close any balanced tree into a tree-rooted planar map such that the root corner 
stays in the outer face. In particular this can be done by repeating the following local 
closure rule: Starting from the root corner and walking counterclockwise around the 
current map, let / be the first outgoing half-edge that is followed by an incoming 
half-edge h’, and glue the pair (h,h’) into an edge in the unique way that leaves the 
root corner in the outer face. 


Theorem 5.3.6 (Walsh and Lehman [169]) Opening and closure are inverse bijec- 
tions between tree-rooted planar maps with n edges, v vertices and f faces, and 
balanced trees with v vertices and f + 1 outgoing half edges. 
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Figure 5.12 
The orientation induced by a spanning tree, and the ccw-exploration. 


A balanced tree should be viewed as a rooted plane tree that carries on its border 
the contour code of another tree. We will reuse this idea several times. 


5.3.2 Spanning trees and orientations 


Mullin’s encoding explains his convolution formula for the number of tree-rooted 
planar maps, but not the simple product form C,C,,; arising from the Chu- 
Vandermonde formula. In preparation of an explanation of this formula, we first 
“forget about the spanning trees” and reformulate the result in terms of orientations. 


The orientation induced by a spanning tree. Let (M,c) be a rooted map, and let 
T be a spanning tree of M. The orientation of (Mc) induced by T is the orientation 
@r such that each edge of T is oriented toward the root, and each edge e of M\ T 
turns counterclockwise around 7, that is, the unique simple circuit formed by e 
and edges of T is counterclockwise (see Figure 5.12). Equivalently, the map (M,c) 
endowed with @; can be constructed using the closure of Theorem 5.3.6, upon ori- 
enting the edges of the balanced tree toward the root and keeping the orientation of 
the matched half-edges. 

It is instructive to perform the closure incrementally: Starting from the oriented 
balanced tree, close one edge at a time, and observe that at each step no cw-circuit 
can be created. Finally in view of the orientation of the edges of T, the orientation Oy 
is root-accessible, i.e. there is an oriented path from each vertex to the root vertex. 


Proposition 5.3.7 Let T be a spanning tree of a rooted map (M,c). Then the orien- 
tation Gr induced by T is root-accessible and has no cw-circuit. 


The cew-exploration of an oriented map. A key observation now is that on the 
rooted oriented map (M,c, @r), the ccw walk around T can be performed without 
knowing T. Indeed in the ccw walk, each edge is reached for the first time from its 
endpoint if it is an edge of T, and from its origin if is not an edge of T. This implies 
that the orientation alone allows us to decide at the first visit of each edge whether 
it belongs to T or not, and whether the walk should follow the edge or cross it. The 
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resulting ccw walk around the initialy unknown tree T is called the cew-exploration 
of the oriented map M. 

Conversely an elementary case analysis allows us to characterize the rooted ori- 
ented maps on which ccw-exploration produces a spanning tree. 


Proposition 5.3.8 The ccw exploration of a rooted planar map (M,c) endowed with 
an orientation @ without cw-circuit outputs a tree T, and this tree is a spanning tree 
of M if and only if the orientation C is root-accessible. 


Summarizing these results yields the following theorem, first explicitely stated in this 
general form in Bernardi’s PhD thesis. 


Theorem 5.3.9 (Bernardi [29]) Let (M,c) be a rooted planar map. The ccw- 
exploration and induced orientation are inverse bijections between accessible ori- 
entations without cw-circuit of (M,c) and spanning trees of (M,c). 


Earlier instances of the result in the special case of triangulations and 3-connected 
planar maps appears in [148, 105], extensions were proposed in [29, 6]. Observe that 
ccw-exploration is just leftmost oriented depth first search; it might thus be the case 
that this result is known already in other contexts. 


Corollary 5.3.10 The number of rooted planar maps with n edges endowed with a 
root-accessible orientation without cw-circuit is eayen (CC; =C,Cn41- 


Again in this corollary only the convolution formula is obtained bijectively. 

In view of Theorem 5.2.6, it is tempting to define a root-accessible feasible 
function on the set V of vertices of a rooted planar map (Mc) as a feasible func- 
tion a : V + N such that a@-orientations are root-accessible (observe that if one a- 
orientation is accessible then all are since circuit reversal preserves accessibility). 


Corollary 5.3.11 The number of rooted planar maps with n edges endowed with a 
root-accessible feasible function is Litjan CCC; = C,Cn41- 


Further reformulations and extensions of these results in terms for instance of recur- 
rent sand pile configurations, and relations to Tutte polynomials are given in [29]. 


Cew-exploration and opening. already observed, the orientation induced by a 
spanning tree of a map coincides with the orientation induced by the closure of 
Theorem 5.3.6. As a consequence, Theorems 5.3.6 and 5.3.9 can be combined to 
obtain the following corollary, which will be useful when we turn to the problem of 
enumerating rooted planar maps. 


Corollary 5.3.12 Ccw-exploration followed by opening is a bijection between 
rooted planar maps with i vertices and j faces endowed with an accessible orienta- 
tion without cw-circuit and balanced trees with i vertices and j outgoing half-edges. 
Moreover, this bijection preserves in- and out-degrees of every vertex. 
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Figure 5.13 
The blowing of a vertex, a map M, its complete blowing with the split tree. 


5.3.3. Vertex blowing and polyhedral nets 


The last result we shall present in this section is a construction that was first used 
in a particular case by Cori and Vauquelin in the 80’s [90] and that was shown by 
Bernardi 30 years later to finally explain the product formula C,,C,+1 [28]. 


Vertex blowing and split trees. Let (M,c) be a rooted planar map and @ an orien- 
tation of M. The blowing of a vertex v with out-degree k is the operation replacing v 
by a white polygon with k nodes each carrying one outgoing edge and the incoming 
edges that precede it the cw direction, as illustrated by Figure 5.13. The complete 
blowing ~(M, @) of an oriented rooted map (M, @) consists in blowing each vertex 
of M independently (the blowing of the root vertex is performed as if there was an 
outgoing half-edge at the root corner). The map 4(M, @) naturally inherits from @ 
a partial orientation that we continue to denote @: the edges inherited from M are 
oriented while the edges of the white polygons created by blowings are not. 

A key observation here is that, if (7, @) is accessible, then the oriented edges of 
X(M, @) cannot form cew circuits: Indeed if they would form a ccw circuit then this 
circuit would also be a ccw circuit in M. In view of the blowing rule, the vertices 
of M on this circuit would only have outgoing edges on the left-hand side of the 
circuit. But in a ccw circuit the root lies in the right-hand side of the circuit, so this 
contradicts accessibility. 

Assuming moreover that @ is an orientation without cw-circuit, then £(M, @) is 
an accessible oriented map without circuit, that is a tree. 


Proposition 5.3.13 Let (M,c) be a rooted planar map and @ a root-accessible 
orientation without cw-circuit. Then the oriented edges of the complete blowing 
x = X(M,@) form a spanning tree of X oriented toward c, called the split tree of 
M. 


Polyhedral nets. Let us use the split tree to cut the sphere on which the map ¥ is 
drawn. As illustrated by Figure 5.14, this splitting construction yields a polyhedral 
net. In order to be able to reconstruct the map X from the polyhedral net one needs 
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Figure 5.14 
Cutting along the split tree to get a polyhedral net, another representation of the same 
polyhedral net, and its skeleton. 


to specify the way edges should be glued together. This can be done by explicitely 
recording a planar arch diagram as in Figure 5.2, but it turns out to be more conve- 
nient to just keep track of the orientation of the edges of the split tree on the sides of 
the polyhedral net, as shown in Figure 5.14. The folding of the polyhedral net then 
just consists in iteratively gluing pairs of oriented border edges that originate from a 
same vertex until all boundary edges have been matched. 

Let us describe more precisely the oriented polyhedral nets that arise from this 
construction: They consist of an outerface and a collection of black and white simple 
faces called polygons, such that the sides of white polygons are incident only to black 
polygons, and the boundary of each black polygon consists of an alternating sequence 
of sides incident to white polygons and superedges formed of a ccw edge incident to 
the outerface, followed by a (possibly empty) sequence of cw edges incident to the 
outerface. 

An oriented polyhedral net is rooted if a corner incident to a white node in the 
outerface is marked, and balanced if moreover, starting from the root corner and 
turning clockwise around the polyhedral net, its boundary edges form the contour 
code of a planted planar tree (equivalently if there are always strictly more already 
seen ccw edges than cw edges during the tour). 


Theorem 5.3.14 Splitting and folding are inverse bijections between rooted planar 
maps with i vertices and j faces and n = i+ j —2 edges endowed with an accessible 
orientation without cw-circuit, and balanced polyhedral nets with i white bounded 
faces and j black bounded faces and 2n +2 boundary edges. 


Let the skeleton of a balanced polyhedral net be the bicolored tree obtained by 
putting a (black or white) vertex in each (black or white) polygon and joining vertices 
corresponding to adjacent polygons by an edge. In particular the degree of a black 
(respectively white) vertex in the skeleton corresponds to the out-degree of the as- 
sociated vertex of the initial map (respectively to the number of ccw oriented edges 
around the associated face). Now observe that in a balanced polyhedral net with 
2n +2 boundary edges, the 2n + 2 symbols forming the contour code of the planted 
split tree are written sequentially on the 2n +2 boundary edges of the polyhedral 
net in a deterministic way: The kth opening parenthesis of the code is preceeded 
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by ¢;, closing parenthesis if and only if, in cw order around the polyhedral net, the 
kth superedge consists of a ccw edge preceded by & cw edges. As a consequence 
a balanced polyhedral net 27 + 2 boundary edges is determined by the pair formed 
by its skeleton and its split tree. Conversely any pair formed of a rooted bicolored 
plane tree with n+ 2 vertices and a rooted plane tree with n edges yields a balanced 
polyhedral net with 2n + 2 boundary edges. 


Corollary 5.3.15 (Bernardi [28]) There is a bijection between tree-rooted maps M 
with i vertices and j faces and n =i+ j —2 edges, and pairs (t),t2) where t) is a 
(bipartite) rooted plane tree with i black and j white vertices and n+ 1 edges, and 
ty is a rooted plane tree with n edges. In particular the number of tree-rooted maps 
with n edges is CyCyit. 


A nice feature of the above result is that it not only explains the univariate product 
formula C,41C,, but also give an alternate proof of the bivariate formula as Nj, ;Cy, 


! ! . . . : 
ao is the Narayana number of rooted bipartite trees with 


i black and j white vertices. Or conversely, combined with the formula @) CiCj 
obtained by Mullin’s encoding, it allows us to recover Naranaya’s formula. 


where Nj,j = 


5.3.4 A summary and some observations 


In this section we have successively shown that tree-rooted maps are in one-to-one 
correspondence with shuffle of parenthesis words, cubic maps endowed a rooted 
Hamiltonian cycle, balanced trees, root-accessible oriented maps without cw-circuit, 
balanced polyhedral nets and finally pairs of rooted plane trees. 

On the one hand, these results form a coherent and nice chapter of bijective com- 
binatorics that relies on and extends several extremely classical results of the Catalan 
garden [165]. These constructions may seem intrinsically planar: For instance on a 
surface of genus g, the dual of the edges that are not in a spanning tree of a map do not 
form a spanning tree of the dual, and the combinatorics of higher genus tree-rooted 
maps is not very satisfying. Instead a remarkable observation of Bernardi and Cha- 
puy is the fact that many of these constructions extend to higher genus maps upon 
considering covered maps rather than tree-rooted maps: A covering of a map is a 
spanning subset of edges C such that the local cyclic arrangements of edges around 
T forms a map with one face (recall that a planar map is a tree if and only if it 
has one face). This approach leads to various elegant enumerative consequences, as 
explained in [35]. 

On the other hand, these bijections are some of the most fundamental ingredients 
in the search of bijections for rooted planar maps: Indeed a natural approach to the 
enumeration of rooted planar maps is to endow each map with a canonical spanning 
tree such that the family of balanced trees arising from Theorem 5.3.6 or the family 
of balanced polyhedral net arising from Theorem 5.3.14 are simple to describe and 
enumerate. 
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5.4 Counting planar maps 
5.4.1 The exact number of rooted planar maps 


Rooted planar maps. The most striking result about planar maps is certainly the 
fact that the number of rooted planar maps with n edges has a simple closed formula 


2-3" (Qn) 23" 2h 
al ni(n+2)! = (*). G2) 


and in particular satisfies the following linear recurrence 
(n+2)|4| =6(2n—1)|.@_,|, with |4|=1. (5.3) 


Equivalently, the ordinary generating function (gf) of rooted planar maps with respect 
to the number of edges M"(z) = Yye.qr2™) satisfies 


M"(z) =T(z) —2T(z)° (5.4) 
where T(z) is the unique power series solution of 
T(z) =1432T (z)’. (5.5) 


(The equivalence of Formula (5.2) and Equations (5.4)-(5.5) follows from a direct 
application of Lagrange inversion formula [infra, Chapter on Tree].) Another way to 
write the relation between M"(z) and T(z) is 


£ 2M") =20 (0). (5.6) 
z 


More blandly, M"(z) is the unique power series root of the polynomial 
P(M,z) =1—16z+ 182M — 272M”. (5.7) 


In particular M”(z) is an algebraic function over the field Q(z). 

Formula (5.2) was first discovered by W. T. Tutte in a series of papers written 
between January 1961 and February 1962 where he deals with the enumeration of 
various subfamilies of planar maps. Using little more than the simple idea of root- 
edge deletion, Tutte established in [161] a recurrence for the number E(d,do,...) 
of rooted Eulerian planar maps with dj; vertices of degree 2i and e = ));+, id; edges, 
then guessed that these numbers admit the following simple form 


7 2(e!) ie pee ee 
Elda) = Gaye ida ( ) . (5.8) 


He then checked that this formula satisfies his recursion by an incredible computa- 
tional tour de force. Quickly after this, Tutte observed in [162] that the special case 
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e- ‘ts; @ 8: O8- 


Figure 5.15 
Tutte’s root edge deletion. 


dy =n and d; = 0 for all i 4 2 of Formula (5.8), that counts rooted tetravalent planar 
maps with n vertices, gives Formula (5.2) for the number of rooted planar maps with 
n edges in view of the edge map transformation (see page 338). 

A few years later, in [163], Tutte proposed a streamlined version of the root edge 
deletion method he uses to establish recursive decompositions and to translate them 
into functional equation for gfs: He is literally applying there what is now known as 
the symbolic method for combinatorial enumeration [102, Chapter 1]. Let M;(z,y) 
denote the gf of rooted planar maps with a root face of degree j, counted by number 
of edges (variable z) and by number of non-root faces with degree i (variable y,) for 
all i > 1, and let M(z,y;u) = Yjs9u/M;(z,y). Then Tutte’s equation reads 


M(z,y;u) — Dig u/M,(z,y) 
i—2 ’ 


M(z,ysu) =1+z°M(z,ysu) +2) yi (5.9) 


i 
i>1 u 


where each term has a clear interpretation in terms of a decomposition by deletion of 
the root edge, as illustrated by Figure 5.15. 


e The quadratic term accounts for planar maps whose root edge is a separating 
edge: Such a map can be uniquely obtained from a pair of rooted planar maps 
by joining their root corners with a new root edge. 


e The ith term in the sum instead corresponds to planar maps whose root edge 
has a face of degree i on its left-hand side: Such a map can be uniquely 
obtained from a rooted planar map whose outerface has degree at least i— 1. 
Hence the truncated M series. 


Iterating Equation (5.9) as a fixed point equation in the space of formal power 
series, one obtains the first coefficients: 


M(z,y,u) = 1+z? (1+ O(2))? +21 AO = 1 4:20? +01) + O(2) 


1+2(u?+uy1) 
ar = 


= 1+zu?(1+z(u*+uy1))? +21 


1+2(u? +uyi)—1 
rZV2 io Tr £Y3 


1+z(u* +uy1) — (1 +zuy1) 
u 


+0(z3) 


1+2(u? +uy1) +27 (2u4 + 3u3y +0? yj +u?y2 + uyry1 +uys) 
+0(z°). 


One can check that these first terms agree with what can be seen on Figure 5.7, and 
for u = 1 and y; = 1 for all i, with the first values of Formula (5.2). 
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In the case y; = 0 for all odd i, Equation (5.9) is essentially equivalent to the 
recurrence used by Tutte to conjecture and prove Formula (5.8), but no such simple 
formula is known to enumerate generic (i.e. non necessarily Eulerian) rooted planar 
maps with a fixed vertex degree distribution, and Tutte was unable in the 70’s to 
guess the general solution. Instead he observed that for y; = y, his equation rewrites 
as 


ar 
M(z,ysu) = EEDA “) M,(z,y) 
70 \i=0 
= 1+zu’M(z,y;u)? +zyu (ys 1) — uM @y a: (5.10) 


l-u 


and using again a guess and check approach, he was able to solve this equation 
to count maps by number of edges and faces. The resulting number of rooted pla- 
nar maps with k vertices and @ faces and n = k+£—2 edges is a not particu- 
larly appealing triple summation of binomial coefficients [13] but the bivariate gf 
M’ (x,y) = Ye arx" yf = xy(1+M(x,y/x;1)) satisfies an elegant variant of 
Formula (5.4): 


M"(x,y) _ N(x,y)P(x,y)(1 _ 2N(x,y) _ 2P(x,y)) (5.1 1) 
where N(x,y) and P(x,y) are given by the bivariate analog of (5.5) 


N(x,y) = x+N(x,y)? +2N(x,y)P(y), 
y+P(x,y)? +2N(x,y)P(x,y). (5.12) 


v 
as 
& 
x 
=z 

| 


Equivalently, the bivariate analog of Equation (5.6) reads 


2 Mt(s,y) =Ple9). 

Solving Tutte’s equations. The difficulty in deriving the explicit expressions (5.2)- 
(5.5) or their refinement (5.11)-(5.12) from Equation (5.10) lies in the particular role 
played by variable u: Observe indeed that variable y can be considered as an optional 
parameter (setting y = | yields a valid equation that allows us to compute M(z, 1;) 
order by order in z by iteration as above), whereas variable u appears to be necessary 
to get a non trivial equation (setting u = 1 yields instead an a priori useless equation 
involving M(z,y;u) and iM (z,y;u)|y=1 because of the discrete derivative appearing 
in the last summand). Following Zeilberger, the variable u is called a catalytic vari- 
able (in analogy with the catalytic ingredients that are sometime added to allow for 
a chemical reaction to take place), and Equations (5.9) and (5.10) are equations with 
one catalytic variable. 

Starting with Brown [74], methods to solve quadratic equation with one catalytic 
variable without guessing were developed, that allowed in particular Bender and Can- 
field [21], almost 30 years later, to give an essentially complete solution to Tutte’s 
equation. Brown’s method was later turned into a systematic approach to polynomial 
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equations with one catalytic variable by Bousquet-Mélou and Jehanne [55]. Let us 
quote here their main general statement. 

Let K be a field of characteristic 0. Let F (u) = F(z; u) be an element of K[u]|[z]], 
that is, a power series in z with coefficients that are polynomial in u over K. Then the 
following divided difference is well-defined: 


and lim,,9 AF (u) = F’(0) (from now on in this section derivatives are taken with 
respect to the variable u unless explicitely mentioned). More generally the iterated 
application of A yields 


F(u) — F(0) —uF’(0)-... pF) 


ui 


Then the main result of [55] is the following theorem (see also [89] for an earlier 
derivation of the case k = 1). 


Theorem 5.4.1 Let Q(wo,w1,W2,...,Wk3Z) be a polynomial in u, z and the w; with 
coefficients in a field K with characteristic 0. Then the unique formal power series 
solution F (u) of the equation 


F(u) =F(z,u) =20 (F(w), AF(u) APF (w), - A F(u),u;2) (5.13) 


is algebraic over K(z, u). 


For any ip > 2, this theorem directly applies to the specialization y; = 0, i > ig of 
Equation (5.9) and proves that the corresponding series are algebraic. 

The proof of the theorem is in fact constructive, and in principle gives a method 
to derive a system of algebraic equations that determines F (uw) and all the A’F (uw), as 
well as the specializations F(—!) (0). We content ourselves here with the case k = 1, 
which is sufficient to derive Equation (5.7) from Equation (5.10), and refer to [53] 
for a gentle introduction to the general case and to [55] for the full details (see also 
[34, Section 9] for a slight extention allowing F and Q to be rational in uw). 


A derivation of Formula (5.2). The first observation is that up to a slight change 
of variable F(u) = (1+u)(M(1 +) — 1), the simplified Tutte’s equation (5.10) can 
be recasted into the form (5.13) with k = 1, and 

QO(wo,W1,W23z) = (1+ wo)(1 +w2 + wo)? + (1 +w2)?(1 +1). 


The initial trick is to differentiate Equation (5.13) with respect to u to get 


Flu) = 2F"(W)Qi(F(W),AF (w) ae2) + M1 (EW) AF (u). a2) 


— BELO 61 (Fu) AF (u),u:2) +205(F (u) AUF (432) (5.14) 
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Now observe that since Q(wo,w1,W23Z) is a polynomial and F (uv) is a power series 
in z with polynomial coefficients in u, the equation 


U =UQo(F (U), AF (U),U3z) +20; (F(U),AF(U),U3z) (5.15) 
is contractant on the space of power series in z and has a unique power series solution 
U(z) = zQ',(0,0,0;0) + O(z*). Upon multiplying by u and substituting w = U(z) in 
Equation (5.14), all terms in the first line cancel and we obtain a second equation 

AF (U)Q\ (FU), AF (U),U;z) =U"Q5(F (U), AF (U),U3z) 
which can be rewriten, using Equation (5.15) to eliminate Q', as 
AF (U)(1 —2Q0(F (U),AF(U),U;z)) = 2UQ)(F(U), AF (U),U;z). 


Summarizing, the triple of unknown functions (F(U(z)), AF (U(z)),U(z)) is a solu- 
tion of the system of algebraic equations in the variables wo, w; and w2: 


2Q(wo,W1, W232) 
wi = 2(w1Qo(wo,w1,W23Z) + Q5(wo, w1,W23Z)) (5.16) 
z(w2Q(wo,W1,W25z) + Of (wo,W1,W23Z))- 


= 
S 
| 


= 
Ne 
I 


In particular the series F(U(z)), AF (U(z)) and U(z) are algebraic series over K(z), 
and so is 
F (0) = F(U(z)) —U(z)AF (U(z)). (5.17) 


Finally, returning to Equation (5.13), F (uw) is seen to be algebraic on K(z,u). Observe 
moreover that if Q has positive coefficients then the system (5.16) is N-algebraic in 
the sense of [53]. 

Eliminating the unknown functions F (V(z)), AF (V(z)) and V(z) in the system of 
equations (5.16)-(5.17) results in an algebraic equation for F (0) that directly yields 
Equation 5.7 for M(z,0) = 1+ F (0). 


The complete solution of Tutte’s equation. Let us finally state the complete so- 
lution of Equation (5.9), as taken from [21, 59, 55]: the gf M(z,y) = M(z,y, 1) of 
rooted planar maps with respect to the number of edges (variable z), and the number 
of faces of degree i (variable y,) for all i > 1 satisfies 


P) 1 
9, Miz) = 7752(982 — 451) (5.18) 


where S; and Sz are the unique formal power series in z with coefficients in 
Ql[y1,¥2,---] solutions of the system 


S; = t[v°] Y yivt Si re Sp/v)i! 
i>1 
So — t+tv "| ¥yi(vt+S,4+S2/v) 1. (5.19) 


i>l 
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These expressions may look complicated at first sight, and maybe one could be 
tempted to argue that Tutte’s equation (5.9) is more compact. However Equa- 
tions (5.18)-(5.19) should be considered as much more explicit: They have a clear 
tree-gf structure, directly amenable for instance to bivariate Lagrange inversion, or 
to singularity analysis, and they imply that for any non-degenerate finite set Z of 
allowed degrees, the gf of rooted planar maps with vertex degrees in Y with respect 
to the number of edges (variable z) are given by the specialization y; = dj<¢g of the 
above system, and in particular this gf is algebraic over Q(z). 


Tutte equations and matrix integrals. Remarkably Tutte’s results were repro- 
duced in the physics literature [73, 41] using an ingenious representation of the 
all-genus map gf using Hermitian matrix integrals. We shall not discuss here this 
alternative approach because various accessible texts already exist. We only stress 
the fact that there are (at least) two point of views regarding matrix integral represen- 
tations. 

On the one hand they can be used as a convenient shorthand notation for the all 
genus gf of maps viewed as a formal power series, and this point of view leads to 
the derivation of loop equations or Schwinger-Dyson equations that are essentially 
equivalent to Tutte equations (see [96]). These equations are dealt with using clever 
changes of variables and more or less detailed discussions of the possible singulari- 
ties of the resulting expressions. These approaches appear to be analytic variants of 
Brown’s more algebraic approach (a rigorous discussion of this is still missing in the 
literature however). From there many explicit formulas have appeared in the physics 
literature, culminating with Eynard and Orantin’s topological recurrence [95, 99]. All 
these results are closely related to the present enumerative approach and we refer to 
[125, Chapter Matrix] for an elementary introduction, and [97, 58] for more detailed 
discussions. 

On the other hand one can try to give a bona fide matrix integral representation to 
the all genus gf viewed as an analytic function. This is the original point of view of 
[73, 41], which allows them to use more analytic tools to derive explicit expressions, 
like saddle point approximations and orthogonal polynomials. However the validity 
of these representations requires careful discussion [94, 118, 151, 121] and involves 
deep relations with matrix integral theory, which makes this approach less elementary 
to follow from an enumerative point of view. Finally let us refer to older but very 
complete surveys [8, 92] for discussions of the motivations for the study of maps in 
physics. 


5.4.2. Unrooted planar maps 


For unrooted maps on the oriented sphere, the formulas are still explicit, although 
somewhat more involved. In particular the number of unrooted planar maps with n 
edges is 


uy __ 1 rl (3) d+2 rj n+l (—1)" r 


d\n 
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Figure 5.16 
A symmetric map and the construction of the quotient map associated with the rota- 
tion of order 4 with vertical axis. 


where @(x) is Euler’s totient function, giving the number of non-trivial divisors 
of x. 

The form of this formula admits a nice explanation. Generically there are 2n 
choices of root corner for an unrooted map, hence the first term in the right-hand side. 
But in this first attempt, maps with non trivial automorphisms are undercounted since 
they correspond to less than 2n rooted maps each: According to Burnside lemma, to 
obtain the correct formula, one has to introduce a correction term for each pair (M, @) 
where @ is an automorphism of M. 

The exact correction terms to be used arise from a classification of the possible 
automorphisms of planar maps. This classification is best understood in geometric 
terms, using the (non-trivial, but very appealing) fact that any symmetric map admits 
a drawing on standard unit sphere in R? that realizes its automorphisms as isome- 
tries of the oriented sphere, i.e. rotations. Rotations acting on maps are then simply 
classified according to their order, and to the type of cells (vertex, edge, or face) that 
they leave invariant (a rotation fixes exactly the two cells intersecting its axis). 

Once the classification is established, the idea is that each pair (M,@) with @ 
a rotation of order n/d can be constructed from its quotient map M/@, which is 
identified as a map with d edges and 2 marked cells as illustrated by Figure 5.16. 
The construction slightly differs depending whether the rotation fixes an edge or not: 
If not, the two fixed cells are chosen among the v(M) + f(M) = e(M) + 2 vertices 
or faces of M, and this yields the first correcting term; otherwise, depending on the 
parity of n either one or two fixed cells are edges, and this explains the alternating 
second correcting term. 

This elegant approach of unrooted map enumeration via the study of quotient 
maps was pioneered by Liskovets [132] (see also [166]) and extends to count planar 
maps up to sense reversing automorphisms [133] (see also the more recent [135, 137] 
for a streamlined presentation). An alternative approach based on multiply rooted 
maps is due to Wormald [171, 172], building on earlier work of Brown [75], but it 
yields less attractive formulas. 
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5.4.3 Two bijections between maps and trees 


The series T(z) given by Formula (5.5) is closely related to the classical Catalan 
numbers gf, and as a consequence it admits dozens of combinatorial interpretations 
in terms for instance of plane trees, lattice paths or non crossing arch diagrams. A 
natural question that was raised very soon after the publication of Tutte’s results is 
how to use such a classical interpretation of T(z) to explain Formulas (5.2), (5.4) 
or (5.6). Two main interpretations have lead to particularly elegant answers: well- 
labeled trees [90] and blossoming trees [153]. 

Curiously, neither of these two interpretations directly deal with rooted planar 
maps, but rather with quadrangulations and tetravalent maps: Recall indeed again 
that, thanks to the incidence map and edge map constructions of page 338, all the 
above formulas apply to rooted planar quadrangulations with n faces, or to rooted 
tetravalent maps with n vertices. 


The blossoming tree approach. This first approach builds on a variant of binary 
trees to interpret Formula (5.5) and to arrive at a direct explanation of Formulas (5.2) 
and (5.4) in terms of tetravalent maps. 

More specifically, let a blossoming tree of size n be a rooted planar map with 
one face (i.e. a plane tree) with n vertices of degree 4 (the nodes) and 2n + 2 vertices 
of degree one (the leaves) that are colored in black and white in such a way that every 
node is incident to exactly one black leaf. As illustrated by Figure 5.17, the standard 
decomposition of rooted plane trees applied to blossoming trees rooted at a white 
leaf matches Equation (5.5), so that the gf of these white trees is T(z). Similarly, the 
ef of black trees (blossoming trees rooted at a black leaf) is zT(z)>. 

Now the right-hand side T(z) — zT'(z)* of Formula (5.4) can be interpreted as 
the gf of balanced blossoming trees: Select for each unrooted blossoming tree with 
n nodes a canonical way to match its n black leaves to n of its n +2 white leaves, 
and declare balanced the white trees that are rooted on an unmatched leaf of the 
underlying unrooted blossoming tree. The gf of balanced blossoming trees is then 
the difference of the gf of white and black trees. Equivalently the right-hand side 
of Formula (5.2) can be interpreted as follows: A fraction as of the aa white 
trees with n nodes are balanced because among the n+ 2 leaves of any unrooted 
blossoming tree, exactly two are unmatched. 

The bottom line of this approach is that the requested canonical matching can be 
performed in a greedy iterative way that preserves planarity and maintains unmatched 
leaves in the outer face, as illustrated by Figure 5.17: in particular the closure of 
Theorem 5.3.6 describes how to do this. In view of Corollary 5.3.12, closure yields 
in fact a one-to-one correspondence between balanced blossoming trees and some 
oriented almost tetravalent maps with two vertices of degree | in the outer face. 
Upon gluing these two vertices to form a root edge, the correspondence is easily 
recasted as a bijection between balanced blossoming trees with n nodes and rooted 
tetravalent planar maps with n vertices endowed with an Eulerian orientation without 
cw-circuit. 
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Figure 5.17 

(i) An unrooted blossoming tree 7. (ii) The decomposition of white and black trees. 
(iii) The canonical matching of the black and white leaves of T. (iv) The iterative 
closure of a balanced blossoming tree into a 2-leg tetravalent map. (v) The associated 
2-oriented tetravalent maps, and the underlying non-oriented rooted tetravalent map. 
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Now according to Theorem 5.2.6, each rooted tetravalent planar map has a unique 
Eulerian orientation without cw-circuit, and any Eulerian orientation is accessible. 
The bijection is therefore between balanced blossoming trees with n nodes and rooted 
tetravalent planar maps with n vertices. This proves that Formulas (5.2) and (5.4) 
count rooted tetravalent planar maps with n vertices, and, via the edge map construc- 
tion, rooted planar maps with n edges. 


The well-labeled tree approach. This second approach builds instead on some la- 
beled plane trees to interpret Formula (5.5) and involves a double pointing argument 
to explain Formula (5.6) in terms of rooted planar quadrangulations. 

Let a plane tree be well-labeled if its vertices carry positive integer labels that 
differ at most by one along each edge, and the minimal label is | (see Figure 5.18(4)). 
Rooted well-labeled trees are in one-to-one correspondence with rooted embedded 
trees that have integer labels that differ at most by one and root label 0: given a rooted 
well-labeled tree with root label k, decrease by k all labels to get a rooted embedded 
tree, and vice-versa, given a rooted embedded tree with minimal label —¢ < 0, add 
£+ 1 to all labels to get a rooted well-labeled tree with root label 2+ 1. 

As illustrated by Figure 5.18(ii), T(z) is the gf of rooted embedded trees counted 
by their number of edges, or, equivalently, since there are three possible variations 
of labels along each edge, the number of rooted embedded trees with n edges is 
3” times the nth Catalan number mal (). Consequently, this is also the number of 
rooted well-labeled trees with n edges. 

Consider now a planar quadrangulation Q with a marked vertex vg and label 
each vertex v of Q by the number of edges in a shortest path from v to vo (see Fig- 
ure 5.18(iii)). In each face of this labeled quadrangulation, draw a new edge accord- 
ing to the rules of Figure 5.18(iv). As suggested by the example in Figure 5.18(vi)- 
(vii), these new edges form a well-labeled tree 7(Q) spanning all vertices of Q 
except vo: The fact that these edges form a tree follows immediately from the facts 
that the rules are not compatible with the existence of a cycle of new edges (because 
vo can only be on one side of the cycle), and that there are n new edges and n+ | 
non-root vertices in Q. 


Theorem 5.4.2 The construction J is a bijection between planar quadrangulations 
with n faces and a marked vertex and well-labeled trees with n edges. 


The fact that construction 7 is a bijection can be deduced from Theorem 5.3.14. 
More precisely, when (Q,c) is endowed with the geodesic orientation (which is ob- 
viously acyclic and accessible), the splitting of Theorem 5.3.14 yields a polyhedral 
net with quadrangular black polygons, each of which is incident to exactly two white 
polygons (see Figure 5.18(viii)). The local rules of Figure 5.18(iv) are seen to de- 
scribe the two possible local configurations in the construction of the polyhedral net, 
as shown in Figure 5.18(v). These observations are sufficient to prove that .7, as a 
special case of splitting, is injective. To conclude the proof of Theorem 5.4.2, one 
needs to check that folding the polyhedral net described by any well-labeled tree 
yields a quadrangulation endowed with the geodesic orientation from one of its ver- 
tices. This follows from the fact that the labels of the vertices of the well-labeled 
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Figure 5.18 

(i) A rooted well-labeled tree and the associated rooted embedded tree. (11) The 
decomposition of rooted embedded trees. (111) A planar quadrangulation Q with a 
marked vertex endowed with distance labels, and the associated geodesic orienta- 
tion. (iv) The local rules defining .7. (v) Local configurations in the geodesic split- 
ting. (vi) The construction of 7 (Q, vo). (vii) The resulting tree 7(Q, vo). (viii) The 
polyhedral net encoded by 7(Q, vo). 
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trees coincide with the natural geodesic labels on the nodes of the folding tree, as 
shown by Figure 5.18(vi)-(v). 

Since each edge of .7(Q) is drawn in a different face of Q, any local convention 
allows us to map the 4n corners of Q onto the 2n comers of .7(Q) canonically to 
obtain: 


Corollary 5.4.3 There is a 2-to-1 correspondence between rooted planar quadran- 
gulations with n faces and a marked vertex and rooted well-labeled trees with n 
edges. In particular this proves Formula (5.6). 


If vo is taken instead to be the root vertex of the rooted quadrangulation Q, then 
the degree of vo is equal to the number of corners with label 1 in .7(Q), and 


Corollary 5.4.4 There is a bijection between rooted planar quadrangulations with 
n faces and rooted well-labeled trees with n edges and root label I (or non-negative 


embedded trees). 


This implies that the number of rooted well-labeled trees with n edges and root label 
1 is given by Formula (5.2), but it is not immediate to use the corollary the other way 
round because counting directly these rooted well-labeled trees with root label 1 is 
not trivial (a direct proof appears however along with the original description of the 
bijection in [90]). Our presentation follows the reformulation of [154]. 


Going further with bijections. The blossoming tree and well-labeled tree ap- 
proaches have grown from the status of nice bijective alternatives to Tutte’s computa- 
tional method into powerful tools that have led for instance to results about distances 
in maps that are currently out of reach of the other methods: See Section 5.4.6 for 
these developments, and Section 5.5.2 for a further discussion of the combinatorial 
properties underlying these bijections. 

In the meantime, let us conclude this paragraph with the remark that the recur- 
rence formula (5.3) also admits an elegant bijective proof [43], akin to Rémi’s bijec- 
tive proof [149] of the recurrence (n+ 1)C, =2(2n—1)Cy_1 for the Catalan numbers 


Cn = et (n): 


5.4.4 Substitution relations 


The non-separable core of a map and 2-connected planar maps. Following 
Tutte [162], let us call a planar map non-separable if it is either reduced to a single 
edge (which can be a bridge or a loop), or if it is loopless and 2-connected. Let @ de- 
note the set of rooted 2-connected planar maps. The non-separable core of a rooted 
planar map is the largest non-separable submap containing the root. As illustrated 
by Figure 5.19, any rooted planar map decomposes bijectively into its non-separable 
core and a collection of rooted submaps attached to each corner of the core. This 
immediately yields the substitution equation 


M'(z) =1+C(zM"(z)’), (5.21) 
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Figure 5.19 
A rooted planar map and its decomposition into its non-separable core and submaps 
attached to corners. 


where C(t) is the gf of rooted non-separable planar maps counted by edges with 
variable ¢ (recall that the number of corners in a map is twice the number of edges, 
hence the substitution t +> zM"(z)*). 

Equation (5.21) allows us to determine C(t) from our knowledge of M(z): Re- 
call that M(z) is an algebraic function, and consider the polynomial P such that 
P(M(z),z) = 0 as given by Formula (5.7). Then the series H(z) = zM(z)? is a root 
of the polynomial Q(t,z) = Resultant,(P(x,z),f — zx”). Moreover, H(z) = z+ O(z’) 
clearly admits a compositional inverse Y(t) in the space of formal power series (that 
is, the unique formal power series Y(t) such that H(Y(t)) = 1). This series is alge- 
braic as well as satisfying Q(t,Y(t)) = 0. Finally letting z= Y(t) in P(M(z),z) =0 
yields P(C(t),Y (t)) =0, from which we deduce that C(t) is a root of the polynomial 
R(x,t) = Resultant, (P(x,z), Q(t,z),z): explicit computations give R(C(t),t) = 0 with 


R(C,t) =C? +2C? + (1— 184)C + 2727 — 2¢. 


Alternatively the computation can be made using the parametrization (5.4)-(5.5) to 
obtain directly a parametrization of C(t) as 


C(t) =2B(t)—3B(t)? where B(t) = 1/(1 —B(t))*. (5.22) 


The number of rooted non-separable planar maps with n edges follows using La- 
grange inversion formula, 
4 1 3n—3 
G.l|=—- : 
[nl 2n 2n—1 ee 


The previous discussion is easily adapted to take into account the number of 
vertices, and this yields the refined formula 
(2i+ j —2)!(i+2j —2)! 
(2i—1)1(27—1) lj! ? 


Misi j41| = 


for the number of rooted non-separable planar maps with i+ 1 vertices and j+ 1 
faces. 
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Polyhedral graphs and 3-connected maps. Following Tutte’s steps [162], one can 
go further and define the 3-connected core of a map: This requires some extra care 
to classify maps without a 3-connected core. The outcome of his analysis, which we 
do not reproduce here (see also [infra, Chapter on Planar Graphs, Section 2]), is that 
any 2-connected map belongs to one of the following three subsets 7”, Y, or H, 
where 


e the set .Y of maps that are serial product of maps of Y or #, 
e the set Y of maps that are parallel product of maps of .Y or #, 
e the set # of maps that have a non-trivial 3-connected core. 


Then, by definition of .Y and Y, the gfs S(t), P(t) and H(t) are determined as 
rational functions of C(t) by the system of equations 


C(t) = S(t)+P(t)+H(t) 
_  _P(t)+A(t) 
S(t) = Fire (5.23) 
S(t) +A (t 
Pt) = spay 


Finally, rooted non-separable maps that have a non-trivial 3-connected core can be 
related to rooted 3-connected maps by a substitution scheme: Each such map is in- 
deed uniquely obtained from a rooted 3-connected map in which each non-root edge 
is replaced by a rooted 2-connected map (the replacement operation of an oriented 
edge e by a rooted map N in a map M consists in identifying the endpoints of e and 
of the root of N and removing these two edges). If G(z) denotes the gf of rooted 
planar 3-connected maps counted by non-root edges (variable z), then the resulting 
substitution equation reads 
H(t) =G(C'(t)) 


where C1 (t) = iC (t) —2 is the generating function of rooted planar 2-connected 
maps counted by non-root edges, C(t) given by Formula (5.22) and H(t) by Equa- 
tions (5.23). As for Equation (5.21), this substitution equation determines G(z) in 
terms of C(z). Again, as shown by Mullin and Schellenberg [145], the system of 
equations can be refined to take into account the number of vertices, and we directly 


state the bivariate result as 


Theorem 5.4.5 (Mullin-Schellenberg [145]) Let U(z,x) and V(z,x) denote the 
unique pair of formal power series solutions of the system 


U=x(1+V)? and V=y(1+U)?. 


Then 


2 2 
1 1 ; (1+U)*(1+V) ) (5.24) 


G(x,y) =x"y" 
(9) (oo I+y (1+U+V)3 
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Ten steps to planar graphs. By substitution we went from 1-connected to 2- 
connected and to 3-connected maps. The enumeration of 3-connected planar maps is 
a great achievement from the point of view of enumerative graph theory because of 
Whitney’s theorem: A vertex labeled 3-connected planar graph essentially has only 
one embedding as a vertex labeled 3-connected planar map. We already argued that 
the exponential generating series of edge labeled planar maps is, up to a derivative, 
the ordinary generating series of rooted planar maps. In the 3-connected case, a sim- 
ilar relation can be devised for vertex-labeled planar maps so that Equation (5.24) 
essentially yields an exponential generating series for labeled 3-connected planar 
graphs. As shown in Chapter 6 of this book, substitution relations, now written for 
planar graphs instead of planar maps, can be used the other way round to obtain suc- 
cessively from (5.24) the generating series of labeled 2-connected planar graphs, of 
labeled connected planar graphs, and of planar graphs. 

This up and down approach to the enumeration of planar graphs was first ex- 
plicitly proposed by Liskovets and Walsh [136] in the form of a ten step program to 
count unlabeled planar graphs. 


5.4.5 Asymptotic enumeration and uniform random planar maps 


Asymptotic formulas. From the asymptotic counting perspective, the exact results 
of Sections 5.4.1 and 5.4.4 have direct simple consequences: When n goes to infinity, 
the number of rooted planar maps with n edges satisfies 


| My| = c- prin? . (1+ 0(1/n)) (5.25) 


where Po = 12 and c is an explicit positive constant, and for any fixed 6 € (—5,4), 
the number of rooted planar maps with n edges and k vertices satisfies 


|Mnra| =cspan(1+o(1)), for k= [n(4—6)] (5.26) 


where again cs and Ps are positive constants that have explicit expressions in terms 
of 5 [22], with ps < po = 12 for all 6 40. Asymptotic enumerative results on rooted 
planar maps are intimately related to the study of the random variable M,, with uni- 
form distribution on the set of rooted planar maps with n edges. From this point of 
view Formula (5.26) can be restated as a Gaussian local limit law with linear variance 
for the number of vertices of My, using the expansion Inps/py = —a@67(1 + 0(1)) 
near 6 = 0, where & is a positive constant (see [infra, Chapter on Planar Graphs, 
Section 3] for similar results in the case of planar graphs). 


Random maps and face degrees. The root face degree is a natural parameter of 
M, to study: As shown by Equations (5.9)-(5.10), the root face degree d¢(M) plays a 
distinguised role in Tutte’s decomposition, and it follows from his approach that the 


series 
M(zuy= 20M) yay (M) 
Mem” 
is algebraic and in fact fairly explicit. As a consequence d¢(M,,) and by duality 
d,(M,,) are quite well known: As n goes to infinity, the degree has a discrete limit law, 
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D(k) = limy5..P(d¢(Mn) = k) > 0, with exponential decay with k. More precisely, 
as k goes to infinity, 
Ds(k) ~ c'(k/m)'/?(5/6)* 
—Soo 


where c’ = /10/20 [108]. Upon selecting uniformly a second root corner at random 
in M,, one realizes that the root face degree distribution is the distribution of the 
degree of the face bordering any random edge in M,,. This rerooting trick makes it 
possible to derive results about the general local structure of large uniform random 
maps: For instance [111], the maximum face degree satisfies for large n, 


Inn — 5InInn 
In(6/5) 


’(max_deg(M,,)) = O(1) 

with a variance of order O(Inn). More precisely, for values of k close to the above 
expected value, the numbers of faces of degree k behave like independent Poisson 
random variables. 


Submaps and asymmetry. Another example is the number ny(M) of induced 
copies of a plane submap H in M, (by an induced copy of H in M we mean a simple 
cycle C such that the vertices, faces, and edges inside and on C form exactly a copy 
of H). In general for a given H there exists a constant c such that the probability that 
ny(M,,) < cn is exponentially small: In other terms, the number of copies of H in 
M,, is almost surely linear. Under further technical restrictions on H, the random vari- 
able ny(M,,) is conjectured to have a mean, a variance, and a Gaussian fluctuation 
(but this is only proved for restricted classes of maps like 3-connected triangulations 
[112]). The almost sure linearity property implies in turn by a remarkably elegant 
argument [150] that among rooted planar maps with n edges, the proportion of maps 
having a non trivial automorphism (of unrooted planar map) is exponentially small 
(essentially because an automorphism must fix the relative orientations of a large 
number of copies of any given asymmetric submap #). In particular this allows us to 
recover independently of Formula (5.20) the asymptotic number of unrooted planar 
maps as 


|i = | Mn |/(2n)- 1 + 0(€")) 


for some positive constant € < 1. It implies furthermore that the random variable U, 
with uniform distribution on unrooted planar maps with n edges behaves asymptoti- 
cally like M,, at least for parameters that are independent of the root and polynomi- 
ally bounded. More precisely if p is an integer-valued parameter of unrooted planar 
maps such that p(M) < c-e(M)® for some @ > 0 and c > 0, then the total variation 
distance between the distribution of p(U,,) and p(M,,) is small: 

)EIP(p(Un) =k) — P(p(Mn) = k)| —>0. 

k 


n—-oo 


This gives an a posteriori further justification of the fact that the literature focuses 
on the slightly easier problem of enumeration of rooted maps. 
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Another interesting consequence of the submap density results is the possibility 
to prove a 0-1 law for the first order logic on planar maps, as pioneered by Bender, 
Compton and Richmond [23]: Roughly speaking one proves that any property ex- 
pressible in this framework has a probability to be true on M,,, which tends to 0 or to 
1 as n goes to infinity. 


Separation properties. Continuing with properties that are accessible via explicit 
bivariate enumeration, another fundamental problem is that of the existence of small 
cuts in random maps. This problem was first raised in mathematical physics, where 
M, appears as a relevant model of 2d quantum geometry: In this context, the random 
map is viewed as a random discretized surface, a 2-dimensional quantum universe 
[8], and a natural question is whether this random universe is expected to branch 
into several almost independent parts. One way to formalize the question (with two 
variants) is to ask about the existence of a vertex v in M,, whose removal produces at 
least two components and (i) one of the components is a tree of size at least k, or (ii) 
both components have size at least k. 

In the first variant (i) the probability that there is such a tree-cutting vertex is 
bounded by the expected number of tree-cutting vertices, itself bounded by 


= pe ; yee Aloe 


» 3 i5/2(n—i)5/2 12 


i=k 


a3 hk, 


where we have used the asymptotic formula (5.25) and the classic approximation 
C, © 4"n~3/? for the number of rooted plane trees with n edges. In particular the 
above bound is exponentially decreasing with k and one can prove that the size of 
the largest induced subtree in M,, is @(Inn). The precise analysis is made possible 
by the existence of a substitution scheme analog to those of Section 5.4.4: Removing 
all induced subtrees from a map M (or equivalently removing iteratively all vertices 
of degree one that are not incident to the root edge) yields its 1-core C!(M), which is 
a rooted planar map without non-root vertex of degree 1. This decomposition yields 
the gf of rooted planar maps counted by number of edges (variable z) and by number 
of edges in the 1|-core (variable u) 


Mr(z,u) =1+C! (uzT (z)”), (5.27) 


where C!(z) is the gf of rooted planar maps without non-root vertex of degree 1, and 
this equation for u = 1 determines the gf C!(z). Then the singular analysis of the 
bivariate substitution scheme (5.27) in the sense of [102] makes it possible to show 
that C'(M,,) has expected size an for some constant @ with Gaussian fluctuations in 
the range n'/2. In the quantum gravity literature, this situation received the evocative 
description of that of a unique mother quantum universe from which logarithmic 
size tree-like baby universes emerge. 

In the second variant (ii) the probability that the separation is possible is bounded 
by 
"—k OnMM,_; n/2 yil2 _3/0 
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This suggests, and it can actually be proved [110], that at most one 2-connected 
component of M.,, has linear size (the mother universe) and the second largest com- 
ponent (the largest baby universe) has size k = @(n/>). The substitution Equa- 
tion (5.21) refines into the bivariate equation 


M(z,u) = 1+C"(uzM(z)?) 


whose analysis shows more precisely [16] that the largest non-separable component 
C*S(M,,) has expected size n/3 with fluctuations in the range n2/> given by a stable 
law of index 2/3. 

The qualitative difference between variants (i) and (ii) is directly explained by the 
fact that plane trees have a strictly lower growth constant than planar maps. Equiv- 
alently, at the technical level, the difference is between subcritical and critical com- 
positions of gfs (see [102, Chapter VI.9]). In both cases, the existence of a bivariate 
substitution scheme to describe the decomposition of M,, into a mother universe and 
baby universes implies that the mother universe, conditionally to its size, is itself 
uniformly chosen among the corresponding set of maps: C™S(M,,) is a uniform ran- 
dom 2-connected map with m edges, where the number m of edges is a concentrated 
random variable. 

The above discussion concentrates on submaps that can be separated by a unique 
cut vertex, but it can be extended to consider 2-vertex separators, and, qualitatively, 
the results are expected to hold for fixed separating cycles of fixed length (although 
in that case there is some ambiguity in the definition of the mother universe). A 
natural next step is to ask about the length of a shortest cycle allowing the separation 
of arandom planar map into two big regions (i.e. so that there are at least an edges 
on each side of the cycle). The answer to this question requires a detour via the study 
of the intrinsic geometry of random planar maps. 


5.4.6 Distances in planar maps 


The intrinsic geometries of planar quadrangulations and planar maps. Given 
two vertices x and y ina map M, let dy(x,y) denote their distance, that is the minimal 
number of edges in a path from x to y. The intrinsic geometry of M is the metric 
space (Vy,dm), where Vy denotes the set of vertices of M. In order to study the 
intrinsic geometry of M,, a natural first step is to consider the distances between two 
random vertices, or between the root vertex and a random vertex. 

This can be done thanks to the two above mentioned bijections between maps 
and trees. In particular our presentation of the bijection 7 of Theorem 5.4.2 clearly 
shows that all the distances to the marked point in a rooted quadrangulation with a 
marked vertex are encoded in the labels of the associated rooted well-labeled tree. 
This gives access to the intrinsic geometry of Q,, the r.v. with uniform distribution on 
the set of rooted planar quadrangulations with n faces. As a consequence most of the 
results on distances in the literature are stated for this related but a priori different 
model of random map. Let us temporarily ignore this problem and discuss the results 
in terms of Q,. 
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From a probabilistic point of view the sole fact that labels of rooted well-labeled 
trees can be interpreted in terms of distances in quadrangulations is already extremely 
fruitful: Let T,, denote a random variable taken uniformly at random in the set of 
rooted well-labeled trees with n edges. The random well-labeled tree T,, is obtained 
by shifting the labels of a random rooted embedded tree E,, by their minimum. The 
tree E,, can be constructed in two steps, first taking a uniform random plane tree 
with n edges, and then choosing the label increment on each edge in {—1,0,+1} 
uniformly and independently. As a consequence a uniform random vertex v in E,, is 
typically at distance 0 = @(n'/ >) of the root in the tree (by standard results on the 
height profile of trees) and its label Ay, (v) is a sum of ¢ i.i.d. random variables taken 
uniformly in {—1,0, 1}: it is thus almost surely of order n'/4 asn goes to infinity. It 
can be proved that this is also almost surely the case for the minimal label, so that the 
statement remains true for T,, [85]. In terms of distances in Q,, we conclude that the 
typical distance between two random vertices is of order n'/4. This combinatorial 
derivation of the typical distance in Q,, proposed in [85], confirms earlier semi- 
rigourous prediction of the physics literature (see [9] and reference therein). 


Exact counting results for labeled trees and distances in quadrangulations. At 
the enumerative level, results can be made even much more precise. For all i > 0, 
let 7; be the gf of rooted embedded trees with label strictly larger than —i. Then the 
standard recursive decomposition of rooted plane trees can be refined to write the 
following system of equations for the infinite family of gfs (7;);>0: 


T=14+27,(T1+7;+T7)41) for i>0, and %M=0. (5.28) 


A true miracle is that the system of equations (5.28) can actually be solved exactly. 
As first shown by Bouttier, Di Francesco, and Guitter in [62], 


(i-¥)a-Y"*) 
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where Y = Y(z) and T = T(z) are the unique power series solutions of 
Y=2T7°(14+Y¥+Y*) and T=1+4+327". 


Observe that the series T in this expression is the same as in Equation (5.5): ac- 
cordingly the limit when i goes to infinity of 7; is just T. While Formula (5.29) was 
guessed and checked in [62], it was recently observed by Eynard and Guitter that the 
system of equations (5.28) belongs to a family of equations known to admit explicit 
computable solution in a different context [98]. 

The bijection between rooted well-labeled trees with root label 1 (or non negative 
rooted embedded trees) and rooted quadrangulations implies that M(z) = T;(z) and 
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in agreement with Formula (5.4). The general explicit expression (5.29) has far 
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Figure 5.20 
A quadrangulation Q with distances to a marked vo, the Ambjgrn-Budd rules, and 
their applications to (Q, vo). 


reaching implications for distance statistics on quadrangulations, as discussed in 
[62, 63, 66, 68, 67, 120] (see also the results mentioned below for distances in M,,). 

Finally, let us mention that the combinatorial structure of Formula (5.29) is still 
not completely clear: from the point of view of quadrangulations, an appealing inter- 
pretation of Y and 7;(z) was given in [70] (see also [3]) in terms of slices of quadran- 
gulations, from which the explicit expressions can be understood. However we still 
miss a direct explanation of Formula (5.29) in terms of the natural interpretation of 
Y /zas gf of well-labeled trees with a marked branch whose labels form an excursion. 


Exact results for distances in planar maps. Quite unexpectedly, many of the ex- 
act results for quadrangulations can be easily transfered to planar maps: As already 
indicated above, the classical incidence map transformation does not help much for 
this, but the solution is provided by another bijection between marked quadrangula- 
tions and marked planar maps that was recently discovered by Ambjgrn and Budd 
[7] (see also [65] for a broader discussion of the relation between this bijection and 
well-labeled tree approach). As illustrated by Figure 5.20 this bijection consists in 
applying to a quadrangulation Q with a marked vertex vo the rules opposite to those 
of Figure 5.18(iv). 

The two key properties of this bijection for our purpose are that (i) it preserves the 
distance to vo of the common vertices of the two maps, namely if (M, 9) = AB(Q, vo) 
then do(v,vo) = du(¥, vo), and (ii) the vertices of Q that do not appear in M are local 
maxima of the distance to vo in Q, and they correspond to vertices whose labels 
are local maxima in the associated well-labeled tree. As a consequence of these two 
properties the labels of a random well-labeled tree that are not local maxima describe 
the distances to a random marked vertex in a random planar map. 

Let us illustrate by an example the very explicit results on the intrinsic geometry 
of random planar maps that are derived in [65]. Let us say that the root edge of a 
vertex-pointed rooted planar map M has type (i, /) if the root vertex is at distance 
i of the pointed vertex, and the other extremity of the root edge is at distance j of 
the pointed vertex (by definition |i — j| < 1). Let then R;(z) denote the gf of pointed 
rooted maps with a root edge of type (j —1,j) for j < i. Upon reverting the root, 
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Rj+1(z) is the gf of such maps with a root edge of type (j+ 1, j) for j > i, and 
let S(z) denote the gf of such maps with a root edge of type (j,/) for j < i (for 
consistency with the literature this gf is written as a square). Then the combination 
of the well-labeled tree approach and the AB bijection is that R; and S? can be written 
for alli > Oas 

R)=27Tj41 and S$? =1T/, 


where the 7; are given by the system of equation (5.29). For completeness let us state 
the explicit expressions for the R; and Sj, 
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In particular these explicit expressions allow to compute distance statistics for M,: 
for instance, for any fixed i > 1, the expected number of vertices at distance i of a 
uniformly chosen vertex in M,, is 


baie hie 2 SO ee Betis 
tim £(|{v | dm, (v,Vo)} =i|) = 590 (2! +3) (10% + 30149). 
Similarly the limit for large n of the expected number of vertices in the ball of radius 
i around a uniform random vertex of M,, grows like i*. 


5.4.7 Local limit, continuum limit 


Local limits. A nice way to subsum the above results on the local structure of ran- 
dom planar maps is the following statement: Large uniform random rooted planar 
maps have a local limit, the uniform infinite planar map, which has Hausdorf di- 
mension 4. The first part of this statement means that, as n goes to infinity, for any 
fixed k, the submap of radius k around the root of M, converges in distribution to a 
random planar map Px with radius k, and the family of random planar map (Rx) x>1 
coherently defines a random infinite (but locally finite) planar map P. The second part 
means that in the random infinite planar map P the number of vertices at distance at 
most i of the root roughly grows like i+. This statement was first stated and proved in 
the case of random triangulations and quadrangulations [11, 10, 84], leading to the 
definition of the Uniform Infinite Planar Triangulation and Quadrangulation (UIPT, 
UIPQ), but again, the Ambjgrn-Budd bijection makes it possible to transfer the re- 
sults to general planar maps to define the UIPM. 

A natural way to continue the study of the local properties of large planar maps 
is to concentrate on these limit uniform infinite planar maps. We will however not 
discuss further this direction for at least two reasons: Many of the methods involved 
there escape from the strict range of enumerative combinatorics, and the topic is 
currently evolving at a very fast pace. We refer the reader to the elegant presentations 
of Nicolas Curien. 
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The continuum limit. Instead of concentrating on local properties, one can return 
to the observation that the distance between the root and a random vertex of M,, is 
of order n'/*. The correspondence between quadrangulations and well-labeled trees 
makes it possible to show more precisely that the rescaled profile (average num- 
ber of vertices at distance k of the pointed vertex) and radius (maximal distance 
to the pointed vertex) of uniform random rooted planar quadrangulation with a ran- 
dom marked vertex converge upon renormalizing the distances by a factor n—!/4 to 
functionals of a well-studied continuum random process called the Integrated Su- 
perBrownian Excursion (ISE) or of its variant the Brownian snake [85]. 

The example of the convergence of rescaled simple random walks to the Brow- 
nian motion and that of rescaled simple trees to the Continuum Random Tree then 
suggests that after rescaling distances by such a factor n—!/4, one should look for a 
continuum limit of random planar quadrangulations. This question has attracted a lot 
of attention in the last few years, that has culminated with the proof of the existence 
and uniqueness of such a continuum limit, the Brownian planar map [129, 140]. 
A key ingredient underlying these results is the fact that the bijective correspon- 
dence between well-labeled trees and pointed rooted maps is robust enough to go 
through the process of taking a continuum limit: The only currently known explicit 
construction of the Brownian planar map consists indeed in starting from the con- 
tinuum limit of rescaled uniform random well-labeled trees of size n when n —- -, 
the above mentioned Brownian snake, and applying a continuum analog of the dis- 
crete bijection to define a metric on this embedded continuum tree. Again the initial 
constructions of [129, 140] were done for quadrangulations, but the Ambj@rn-Budd 
correspondence makes it possible to transfer most of these results to the intrinsic 
geometry of uniform random planar maps with n edges [44]. 

Remarkably it has been proved that the Brownian planar map is a random metric 
space with the topology of the sphere [131, 138]: A consequence of this last state- 
ment is the fact that for any positive @ < 1/2, the shortest cycle on M,, with at least 
an edges on each side has almost surely length @(n'/ 4) when n goes to infinity. Al- 
though, as discussed above, the enumerative results and in particular the bijections 
between maps and trees are fundamental ingredients of construction of the Brow- 
nian planar map, these consequences about shortest a-separating cycles currently 
appear to be out of reach with purely enumerative methods (see however [67] for a 
tractable variant of the problem). Further discussion of these results is thus largely 
out of scope here and we refer to the [130, 141] for a survey of these developments, 
or to [127, 128] for short reviews. 


5.5 Beyond planar maps, an even shorter account 


At this point we are more or less done with general planar maps, and there are (at 
least) four ways to go further: universality, master theorems, maps on surfaces, and 
decorated maps. 
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5.5.1 Patterns and universality 


A first direction to explore is the observation that literally all of the above results 
admit variants for various natural subfamilies of planar maps. In the combinatorial 
literature, patterns in the asymptotic behavior have been observed, and equivalently, 
in the physics and probabilistic literature some critical exponents are expected to be 
universal. 


Pattern in the asymptotic behavior and universality. A first example is the fact 
that for a large collection of subfamilies F of maps for which the asymptotic number 
of rooted planar maps with n edges in ¥ is known, the polynomial growth exponent 
takes the same value y = — 3: let MF denote the set of rooted planar maps with n 
edges in the subfamily .¥, then 


| Oi" | ~ expen? (5.32) 


for all admissible values of n, where cz and pg are constants depending on ¥. This 
asymptotic behavior holds in particular for various families of planar maps defined 
by combinations of a finite restriction on the allowed vertex or face degrees (but not 
restrictions on both), a finite restriction on the girth (length of the shortest simple 
cycle), or an irreducibility condition (a map is irreducible if the length of the shortest 
non-facial cycle is strictly larger than its girth), and for some of these families, with a 
further bipartiteness constraint [26, 27, 18, 107, 79, 31]. Similar patterns or universal 
exponents hold for each of the asymptotic/probabilistic statements of the previous 
section about degree distribution [134], absence of non-trivial automorphisms [24], 
separation properties [152, 110, 16], and distances [69]. 

A quite natural way to unify these results is to state the universality of the Brow- 
nian planar map as a continuum limit: For all “reasonable” family .F of planar maps, 
there should exist a constant &g and a definition of the size such that uniform random 
planar maps of size n in ¥ with distances rescaled by a factor agn—'/4 converge to 
the Brownian map in the sense of [129, 140]. The collection of families of “reason- 
able” maps for which this statement has actually been proved is more restricted than 
for the previous ones but it has recently grown quickly to include planar quadran- 
gulations and more generally 2p-angulations, general, simple, and bipartite planar 
maps, and simple triangulations [1, 2, 4, 44], and all the above mentioned families 
are expected to belong to the same universality class. 

A common point of these families is that they are defined by constraints that 
can be checked locally (at finite distance in the derived map). From this point of 
view examples of “non-reasonable” subfamilies are outerplanar maps because the 
existence of a large outerface is not a local constraint, series-parallel maps or stack 
triangulations whose characterization needs to be checked recursively. And indeed 
these families lead to other continuum limits [5, 76]. A different example is that of 
planar maps with a global rotation or reflexion symmetry whose continuum limit 
is expected to conserve the initial symmetry: They do not even satisfy the initial 
asymptotic pattern because, in view of Section 5.4.2, they are in bijection quotient 
maps, that are multiply rooted planar maps. Finally let us observe that too stringent 
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restrictions on both vertex and face degrees cannot either lead to the same universality 
class due to rigidity constraints: For instance triangulations with too many vertices of 
degree 6 have to be constituted of patches of regular triangular lattices. The question 
of whether a family of maps belongs or not to some universality class is a key issue 
in the physics literature on maps, and in particular on decorated maps: Accordingly 
we shall return briefly to this question in Section 5.5.4. 


Parametric families and meta theorems. A different kind of recurrent statements 
in the theory of maps is illustrated by the observation that many families of rooted 
planar maps defined by local restrictions have algebraic gfs. Here the universality 
is more questionable and it is in fact not difficult to design “reasonable” families 
of rooted planar maps that most likely do not have algebraic gfs and yet will sat- 
isfy most other universality properties: The family of rooted planar maps with only 
prime vertex degrees seems an obvious candidate, and several families of non-critical 
decorated maps, as discussed in Section 5.5.4, provide other examples. 

One way to circumvent the difficulty of defining universality is to obtain general 
parametric results of the form of Equations (5.18) and (5.19). Tutte’s equation can 
be generalized to bicolored maps counted by number of faces of degree i of each 
color [55]. Similar parametric equations have also been written for families of rooted 
non-separable planar maps [170]. However for 3-connected maps and other families 
of maps defined by girth restrictions, equations with catalytic variables are harder to 
establish from the root edge deletion approach (see for instance [45] for the already 
quite intricate case of 3-connected maps counted by edges and root face degree). 

General parametric results have instead been obtained by Bouttier and Guitter via 
a far reaching and beautiful extension of the substitution schemes of Section 5.4.4 to 
the gfs of maps with girth g and d-irreductible maps with respect to the distribution of 
face degrees [72, 71]. Through standard transformations like duality, incidence map, 
edge map, and a few others, their theorem encompasses all the known critical substi- 
tution schemes summarized in [16] and allows us to recover easily the enumeration 
of rooted planar 5-connected triangulations of [109]. 

Another way to assert some form of universality could be to resort to logic and 
prove a meta theorem: Let f be a formula of the first order logic on maps (with 
quantifiers on vertices, faces, edges and corner, and adjacency/incidence predicates). 
It is tempting to conjecture that the family .#/ of rooted planar maps M such that 
f(M) is true has an algebraic gf, and that the number of such maps satisfies the 
universal asymptotic pattern above. (The second part of the statement is actually 
more likely to be true than the first, but probably hard to prove independently.) This 
approach is reminiscent of the 0-1 laws briefly discussed in Page 369, but appears to 
requires new technical ingredients. 


5.5.2 The bijective canvas and master bijections 


The initial motivation for looking for bijective proofs for the formulas of Tutte was 
closely related to Schiitzenberger’s methodology according to which combinatorial 
structures with algebraic gfs should admit natural encodings by words of context free 
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languages (or equivalently, they should be in natural correspondence with simple 
families of trees). Accounts of early attempts in this directions are [86, 87]. A recent 
discussion (and partial refutation) of the statement that algebraic gfs should be the 
trace of such natural encodings can be found in [52]. 

In the case of planar maps, most of the known gf algebraicity results do admit 
derivations by bijections with some trees. Rather than universality results, master 
bijections have been proposed [6, 36, 38, 39], that allow us to derive these bijections 
in a unified way. A beautiful outcome of these results is that they make explicit the 
structural properties of maps that make the existence of such bijections possible. 


Blossoming trees and well-labeled trees made parametric. The first extensions 
of the original two bijections of Section 5.4.3 that were proposed were rather ad- 
hoc variants for specific subclasses like bipartite maps [12, 154], loopless, simple 
or irreducible triangulations or quadrangulations [154, 147, 148, 105, 104, 103] (see 
also [91] for an independantly found recursive variant for non-separable maps). In all 
these cases, known formulas were instrumental to help guessing an adequate family 
of balanced trees, which was then used to design a bijection. 

A first parametric extension of the blossoming tree approach was to the case of 
planar constellations in [56]: Although no previous enumerative results were avail- 
able for this case, conjecturing a formula was again a preliminary to the design of the 
right family of trees. It is yet a first example of a result that is actually much harder 
to derive from Tutte’s root deletion approach [55] than from bijections. 

Similarly, while basic blossoming trees and well-labeled trees are natural combi- 
natorial interpretations of Equation (5.5), parametric extensions were build as inter- 
pretations of the system (5.19). This approach was then extended further to bicolored 
maps counted by number of faces of degree i of each color [57], building on the enu- 
merative indications given by partial enumerative results for trivalent and tetravalent 
maps [60]. With this exemple the paradigm started to change and the bijective ap- 
proach yields new formulas that were not even conjectured before. 


A structural result about orientations. This shift of paradigm lead to the question 
of understanding under what conditions bijections between maps and trees can work. 
In view of Section 5.3, the bijections of Section 5.4.3 are seen to rely on identifying 
an easy to enumerate family of decorated trees whose closure yields the expected 
family of maps by a specialization of the bijections of Theorems 5.3.6 and 5.3.14. 
Conversely, given a family of planar maps, these bijections rely on identifying the 
right notion of canonical spanning tree for which these theorems lead to a simple 
family of decorated trees. A first step is Theorem 5.3.9, which suggests that in order 
to find the right canonical spanning trees, one should look for accessible orientations 
without cw-circuit. This in turn is made easier by Theorem 5.2.6, which asserts that 
it is sufficient to look for an accessible feasible function on our family of maps. 

The key structural result is now the fact that the existence of feasible and acces- 
sible a-functions is a natural graph theoretic property, expressible in terms of girth 
conditions. A most general result of this type is given by Bernardi and Fusy in [39] 
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for what they call fittingly charged hypermaps. We only quote here a subcase of 
their classification taken from [36, 38]: 


Theorem 5.5.1 A planar d-angulation has girth d if and only if it admits an acces- 
sible (d +2)/d-fractional orientation. 


The exact definition of fractional orientation is out of the scope of this text, but the 
key point is that the theory of @-orientations extends to these fractional orientations, 
and in particular the fact that there is a unique such orientation without cw-circuit. It 
suggests that, at least for those classes of maps, the quest is over: Instead of guessing 
the right family of tree from enumerative formulas, one can start from the naturally 
associated feasible a function and work the other way round. 


The bijective canvas. The resulting canvas for bijection between maps and trees 
is the following: Start with a subfamily ¥ of rooted planar maps defined by degree 
and girth conditions. 


1. Reformulate the degree and girth conditions in terms of the existence of some 
generalized @-orientations (using e.g. Theorem 5.5.1). 


2. Fora given a, the set of generalized @-orientations on a given map has a lattice 
structure, and in particular there is a unique such orientation without cw-circuit 
(Theorem 5.2.6). 


At this point there are two options: The first option generalizes the blossoming tree 
approach and was formalized in [6] (see also [155]). 


3a. Use ccw-exploration as in Section 5.3.2 to obtain a canonical spanning tree T 
of the map M, and check that external edges can be encoded as decorations 
of this tree T in such a way that the resulting set of decorated trees can be 
described by local rules, directly inherited from the degree and a-constraints. 


In the case of tetravalent maps, we have already seen that this approach directly yields 
the blossoming trees of Section 5.4.3: A planar map is tetravalent if and only if it 
admits a 2-orientation (a particular case of Euler characterization of Eulerian maps); 
in particular a tetravalent map admits a unique 2-orientation without cw-circuit; the 
cw-exploration of this orientation yields a spanning tree whose opening has vertices 
of degree 4 with in and out-degree 2; by construction, apart from the root vertex, 
each vertex has one outgoing edge toward the root, so that the other outgoing edge 
must be a dangling half-edge, and each vertex has two incoming edges or half-edges: 
This provides, without guessing, the definition of blossoming trees of Section 5.4.3. 

Another example is the case of simple triangulations (that is, triangulations with 
girth three): As first shown by Schnyder [158], a planar map is a simple triangulation 
if and only if it admits a 3-orientation; in particular a simple triangulation admits a 
unique 3-orientation without cw-circuit; the cw-exploration of this orientation yields 
a spanning tree whose opening has two dangling outgoing half-edges per vertices; the 
face degree condition implies that incoming half-edges are not necessary to perform 
the closure; the resulting family of blossoming trees is exactly the family of plane 
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trees such that each inner vertex is incident to two leaves and this allows us to recover 
the bijection first proposed in [146]. 

The second option generalizes the well-labeled approach and was formalized in 
[36, 38, 39]. 


3b. Use vertex blowing and splitting as in Section 5.3.3 to obtain a balanced poly- 
hedral net M, and again, check that, apart from the fact of being balanced, all 
the constraint on the resulting set of polyhedral nets can be described by local 
rules inherited from the degree and a@-constraints. 


In the case of quadrangulations, we have already seen that using the geodesic 
distance to orient the map directly yields the bijection of Section 5.4.3 with well- 
labeled trees. The extension of the bijection to general bicolored maps, known as 
Bouttier, Di Francesco, Guitter’s bijection [59] is recovered as well using the ori- 
entation induced by oriented geodesics and a similar local analysis of the possible 
configurations around black and white polygons: the mobiles of [59] naturally arise 
as simplified descriptions of the resulting polyhedral nets. 

This second approach yields the currently most general master theorem [39]: It 
extends in particular to hypermaps with ingirth d (that naturally generalize maps 
with girth d) and in this context gives a very general set of parametric equations for 
hypermaps with ingirth d and outerface of degree d counted by degrees of black and 
white faces, that extends Equations (5.18)-(5.19). As already mentioned the most 
general statement involves some fittingly charged hypermaps that arise as the nat- 
ural setting to which the technics of proof underlying Theorem 5.5.1 apply. 

Finally we should mention that a third bijective canvas arises from the extension 
of the substitution scheme approach proposed in [72]: This last approach has the ad- 
vantage that it allows to perform non-trivial but directly N-algebraic decompositions 
for planar maps with a boundary, as first proposed in [70, 3]. 


Bijections with other combinatorial structures. Tutte’s formula (5.2) for the 
number of rooted planar maps with n edges, and its variants for rooted non-separable 
planar or triangulations have such a simple closed form that one should expect a 
priori unrelated combinatorial structures to be counted by the same numbers. This 
is indeed the case and bijections have been devised to explain some of these co- 
incidences, in particular in the study of permutations with forbidden pattern (see 
[117] and references therein). A higher level “explanation” is that pattern avoiding 
permutation tends to admit natural decompositions leading to polynomial equations 
with catalytic variables, and we have seen that the simplest of those equations count 
maps. Exhibiting structurally identical recursive decompositions for two equinumer- 
ous families of combinatorial structures is a natural way to get a (recursive) bijec- 
tion and this approach has been fruitful in this context (see [46] and the references 
therein). Another remarkable example is that of Baxter permutations and intervals in 
the Catalan and Tamari orders (see [106] and reference therein). 
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5.5.3 Maps on surfaces 


The third way to go is to move to maps on surfaces, that is, map on the torus, or 
more generally on an oriented or non oriented surface of genus g. The main idea is 
that, mutatis mutandis, many results have analogs for maps on oriented and also on 
non-oriented surfaces but actual statements are more complicated, and proofs much 
more technical. 


Exact and asymptotic counting. A particularly nice result is that Formula (5.25) 
is replaced for rooted map on orientable surfaces of genus g by the more general 


| Mg | = c+ T- 12"). (14 0(1//n)), (5.33) 


where the limit is to be taken at g fixed and n going to infinity [19] (for similar 
results in the physics literature see references in the survey [92]). This result is a 
consequence of a more technical statement giving the general form of the gf of these 
maps [19, 20, 14]: There exists a family of polynomials P,(x) such that 


P.(T) 
(2—T(z))?8" 


As usual, this statement can be refined to take into account the number of vertices 
[22]. Earlier statements for small values of g were already given in [73, 41], and 
probably the general form of the result was known to the physics community in 
the early 80’s as well but we could not trace the reference. Recently, Eynard and 
Orantin have proposed a more explicit expression of the series M8 (z) as a finite sum 
of residues of rational functions in T(z) indexed by some simple trivalent graphs, 
which they derive from an explicit topological recurrence [95, 99]. The statement 
of this result requires too many notations to be reproduced here. 

But maybe the most surprising result is the following simple quadratic recurrence 
recently discovered by Carrell and Chapuy [78]: 


(2n —3)(2n —2)(2n— 1) 


M8 (z) = where T(z) =1+32T(z)’. 


(n+1)M8 = 4(2n—1)M8_, + 5 Me; 
43) Y (2k-1)(2€-1)Mi_,M/_,, (5.34) 
k+l=ni+j=g 
kKe>1 i,j>0 


for n > 0, g = 0, with initial condition M = 1, and Mj =0 for n <0 or g <0. This 
recurrence even admits a barely more complicated refinement for the number Me j of 
rooted planar maps with i vertices and j faces on a surface of genus g: 


+MS,_ ne (2n Sen” 2)(2n— 1) 
caee 2 oy Yi (2m —1)(2ny - 1) MF MB, (5.35) 


ijl 2,J2? 
thsi fitjo=J, 81+ 82=8 
i221 f,,j221 81,8220 


oI 
(n+1)Mi, = 2(2n — 1)(MF Mi; 


Raj 


for i, 7 = 1, with the initial conditions that Mi, =Oifi+ j+2g < 2, that ifi+j+ 
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2g = 2 then Me, = 1,(;,;)=(1,1)}, and where we use the notation n =i+ j+2g—2, 
ny =i, + jy +2g) — 1, and ng = i2 + jo +290—-1. 

While Formula (5.33) arises from (non-trivial) refinements of the gf technics tra- 
ditionally used to study planar maps, it is worth observing that the proof of For- 
mula (5.34) requires more algebraic ingredients based on the encoding of maps in 
terms of permutations briefly used in formalizing Definition 5.2.2 (see [114, 78], the 
earliest reference we could find is [123] in the combinatorial literature and [41, Ap- 
pendix 6] in the physics literature, both in the case of unicellular maps). Another 
remarkable outcome of the approach of [114, 78] is the possibility to derive from it 
detailed asymptotics for the constant tT, [25]. Analogous results for non-orientable 
surfaces have also appeared [19, 107, 77]. 

Unrooted maps in higher genus surfaces have also been considered, see [167] 
and reference therein for exact results, and asymptotic asymmetry results like For- 
mula (5.4.5) are in fact valid on higher genus surfaces as well [24]. 


Universality again. As in the planar case, the asymptotic results are valid not only 
for the family of all rooted planar maps but the pattern holds for various subfamilies: 

| Mgnl = CF Tq: (P#)"-(Oten)2@) (1+ O(1/n)) (5.36) 
where cz, P.¢ and Q@¢ depend on the family, but the dependency in the genus is 
entirely controlled by the constants T, and fg = 3(g — 1) that already appear in For- 
mula (5.33). All the families of maps satisfying the pattern (5.32) are expected to 
satisfy (5.36) as well on higher genus surfaces, but this is currently proved only for 
fewer families (in particular for those defined by one of the aforementioned restric- 
tions: degrees, girth, or irreducibility) [107, 79]. 

Another general observation is the claim by Eynard, that map gfs should in 
general satisfy a topological recurrence parameterized by the so-called spectral 
curve [97]. This statement is more general than the algebraicity and asymptotic pat- 
tern (5.36) since the topological recurrence can be written for gfs of critical decorated 
maps as well as for series arising from algebraic geometry problems. For the numer- 
ous developments around this recurrence we refer to the forthcoming book [97]. 


Bijection for maps on surfaces. Regarding bijective proofs, only the approach 
via well-labeled trees currently extends to higher genus maps: it yields in particu- 
lar a correspondence between rooted maps on a surface of genus g and well-labeled 
unicellular maps of genus g (a map is unicellular if it has only one face). Com- 
bined with decompositions a la Wright, it yields a combinatorial explanation [83] 
of the occurence of the exponent 3(g — 1) in Formula (5.33). A closer relation with 
well-labeled trees with several marked points was given by Chapuy in [80] using 
a remarkable bijective decomposition of unicellular maps [81]. Another remarkable 
extension to higher genus is Miermont’s result for maps with several marked points 
[139]: In particular as discussed in [65], this result unifies the well-labeled tree and 
the Ambjgrn-Budd results. 

As in the planar case, the bijective approach allows us to keep track of distances in 
quadrangulations. Until now exact distance enumeration results like Formulas (5.30)- 
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(5.31) have been obtained only in the case g = 1, [119], but the approach has allowed 
us to show that in general typical distances remain in the order n'/4, [83, 80], and it 
has opened the way to the construction of continuum limits of random quadrangula- 
tions on surfaces [42]. 

It is instead an open problem to give combinatorial interpretations of the Eynard- 
Orantin topological recurrence, or of the Carrell-Chapuy recurrences (5.34) or (5.35). 
The later problem is particularly intriguing: Each term in the recurrence has a clear 
combinatorial interpretation, and the special case j = | is the celebrated Harer-Zagier 
recurrence for the number €,(”) = Ms +12.) Of rooted unicellular maps with n edges 
and genus g, for which an elegant bijective proof was recently proposed by Chapuy, 
Feray and Fusy in [82], building on the earlier construction of Chapuy [81]. 

It is worth mentioning the fact that there is a parallel story of unicellular maps, 
starting with the root deletion method [168, 169, 170], continuing with characters 
of the symmetric group [123, 116, 146] and matrix integrals [122] to arrive to more 
combinatorial methods [126, 115, 156, 40, 32]. 


5.5.4 Decorated maps 


Finally the fourth direction to discuss is the study of decorated maps. The decoration 
of a map can be a coloring of its vertices, a spanning tree, a spin or loop configuration, 
an orientation, etc. Of interest is then the gf of maps endowed with such a structure, 
or more generally the weighted gf where each decoration is given a weight and the 
weights are summed over all pairs (map,decoration). We briefly discuss at the end 
of this section some motivations for this study arising from physics and probability, 
but we concentrate on counting results that have been obtained in the combinatorial 
literature, either in the form of explicit counting formula or of equations for gfs. 


Exact counting results by direct decompositions. As discussed in Section 5.3, 
probably the first model to have been dealt with is that of cubic maps endowed with 
a Hamiltonian cycle, which were enumerated by Tutte in [160] even before counting 
planar maps. Spanning trees on general maps were counted later by Mullin [144] 
who realized that there is a simple correspondence between these two models (see 
also Theorem 5.3.4). Other models like bipolar orientations [17, 48, 106], realiz- 
ers [47, 33, 106], or Schnyder decompositions [37] share with spanning trees and 
Hamiltonian cycles the property that they enjoy quite explicit formulas and bijec- 
tive proofs, even though their generating series are non algebraic. We include in this 
list the bijection of [30] although it deals a priori with the family of cubic planar 
map without decoration but the proof here uses the fact that each such a map admits 
2” spanning trees and then deals with tree-rooted maps. We believe that all these 
results await for a unifying master bijection mapping rooted planar maps with an 
a-orientation onto some bidimensional walks, maybe extending the master bijection 
between rooted planar maps endowed with a minimal c-orientation onto decorated 
trees as discussed in Section 5.5.2. 
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Trees and Tutte’s equations for decorated maps. Another family of results is that 
of dimer models, hard particles, and Ising models on planar maps that admit bijective 
proofs based on the blossoming or well-labeled approach [61, 57, 64], and enjoy 
algebraic generating series. Most of these results were first obtained via the matrix 
model formalism, and several have been rederived via Tutte root-deletion method 
before getting bijective proof with trees. 

The enumeration of planar maps weighted by the Tutte polynomial (also known 
as the Potts model on random lattices) plays a particular role in this list, as it contains 
many of the above previously solved models: As first shown by Tutte for the special 
case of well g-colored triangulations in a long series of 10 papers spanning 20 years 
of research ({164] and ref. therein), this model is still amenable to the root-deletion 
approach but the equation is amazingly difficult to solve. The resulting gf is non- 
algebraic but it satisfies an explicit differential algebraic equation of order three. 
Tutte’s solution remained as an isolated artifact for almost 25 years until Bernardi and 
Bousquet-Mélou were able to distillate and extend his approach to the general Tutte 
polynomial on triangulations and to deal with non-trivial variants of the problem 
on rooted planar maps (see [34, 53] and reference therein). As opposed to the now 
well-understood polynomial equations with one catalytic variable that is associated 
to standard Tutte equation (see Section 5.4.1), much is still to be understood on the 
class of polynomial equations with two catalytic variables that are involved in these 
decorated models. 


Exact results from substitution schemes. Other models have been dealt with by 
variants and generalizations of Tutte’s substitution schemes that we discussed in Sec- 
tion 5.4.4. Using a refinement of this approach Sundberg and Thistlewaite obtained 
the gf of diagrams of open prime alternating links [159] and the asymptotic number 
of prime alternating links then follows from an asymmetry analysis [124]. Zinn- 
Justin and Zuber [173] extended the approach to 2-colored links, in an attempt to 
attach the problem of enumerating prime alternating knots (see also [157]). Tutte’s 
substitution scheme allows us also to deal with the Ising model on non-separable 
maps [34] or with orientations on maps with girth or connectivity constraints [101]. 
In all those examples, the main idea is that the decomposition follows the structures 
of the underlying non-decorated map, and the interactions between the decoration 
and the decomposition is local and relatively well controlled. 

In the case of the so-called O(n) of colored loops on maps, Borot, Bouttier, and 
Guitter [51, 50, 49] apply instead the idea of decomposing along the maximal loops 
of the model. By a careful analysis of the possible interface between the core (here re- 
named gasket) and the submaps along these maximal loops, they obtain a functional 
equation that they are not able to solve exactly in general but which is sufficient to 
derive remarkably precise non-trivial asymptotic results. 

Finally let us quote the recent work of Bousquet-Mélou and Courtiel [54], which 
starts this time from a direct, non-recursive, substitution scheme: Rooted planar map 
with a spanning forest can indeed be obtained from non-decorated maps upon in- 
serting spanning trees in vertices with proper weights, as already observed in [64]. 
This approach yields almost directly an equation, but its study is quite difficult and 
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leads to remarkable developments involving differentially algebraic gf and to non 
standard asymptotic expansions for a particular interpretation of the model in terms 
of forested maps weighted by the external activity of the spanning forest. 


Decorated maps in physics and probability. To conclude, let us give a beotian’s 
motivation for the study of decorated map in physics. From the physics point of 
view, we are considering the annealed partition function of a toy model of statisti- 
cal physics coupled to a random lattice. Toy models of statistical physics are usually 
defined on a fixed regular lattice (typically the triangular, square or hexagonal lat- 
tice) and are viewed as microscopic models of matter. The idea to couple such a toy 
model with an irregular lattice arises from quantum geometry: The regular classical 
euclidean geometry is to be replaced by a quantum geometry, that is a distribu- 
tion of probability on a set of irregular geometries: In the pair (map, decoration), the 
map plays the role of the random lattice and it is decorated by the configuration of 
the model. Accordingly, non-decorated maps are considered as a model of pure 2d 
quantum geometry, while decorated maps correspond to 2d quantum geometry 
with matter. We refer to the survey [92] and the book [8] for an introduction to the 
motivations for the study of these toy models on random lattices in physics. 

The physics community has developed a remarkable expertise in classifying the 
toy models into universality classes, and in predicting which universality classes and 
critical exponents are possible. From this point of view the pattern and universality 
statements of Section 5.5.1 are just the tip of the iceberg, corresponding as already 
mentioned to the simplest case of pure 2d quantum geometry without coupling to 
matter. The more general situation of quantum geometry with matter is quite compa- 
rable to that of the study of statistical physics conformally invariant toy models on 
regular 2d lattices, and in fact a tight relation exists between these two areas, first 
discovered by Knitchik, Polyakov and Zamolochikov and now refered to as the KPZ 
relation. Again the interested reader is directed to [92], and to the work of Duplantier 
and Sheffield [93] and Miller and Sheffield [142] for recent developments on these 
interactions, see [113] for a survey. 
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6.1 Introduction 


Many references to date on the enumeration of graphs deal with unlabeled graphs, 
like the monographs of Harary and Palmer [32] and Polya and Read [50]. The main 
tool there is the cycle index polynomial and the goal is to obtain exact expressions for 
the number of graphs of a given kind, like trees or connected graphs. Our emphasis 
in this survey is instead on labeled graphs. One reason is that in many interesting 
cases the number of unlabeled graphs with n vertices is asymptotically the number 
of labeled graphs divided by n!, the number of possible labelings. This is not always 
so, as in the case of trees, since a random tree has almost surely exponentially many 
automorphisms. 

A class of graphs is a set of labeled graphs closed under isomorphism. Typical 
examples of interest are bipartite, acyclic, triangle-free or planar graphs. We use n to 
denote the number of vertices in a graph. Let Y be a class of graphs and Y, the graphs 
in Y with n vertices. The main problem we consider is to compute or to estimate |G,,| 
when 1 — oe. In very few cases we have exact formulas. Here are some examples: 


e The number of labeled graphs is 2(2). This is because each of the CG) edges 
of the complete graph can be chosen independently to be or not be in a graph. 


Likewise, the number of graphs with v vertices and m edges is equal to ((2)), 


e The number of labeled even graphs (all vertices have even degree) is 22°), 
There is a very simple proof of this fact: Given a graph G on n — | vertices, 
add a new vertex labeled n and connect it to all vertices in G of odd degree. 
This gives an even graph on n vertices and it is easy to check that this is in 
fact a bijection. 


e The number of labeled trees is n"~*. This is the well-known Cayley’s for- 
mula, which admits many different proofs. There is no simple formula for the 
number of forests (acyclic graphs). However, the number of labeled forests 
consisting of k rooted trees is (i) me, 


The main emphasis in this chapter is on asymptotic results. In many cases we use 
generating functions, which are of the exponential type since graphs are labeled. We 


n 
start with a basic example. Let G, = 2(2) be the number of graphs on n vertices, with 
the convention that Gp = 1. In order to compute the number C,, of connected graphs, 
we introduce the generating functions 


Gx)=) Gr C(x) = VGH. 


n>0 : n>0 


The exponential formula G(x) = eC“) implies that 


C(x) = log ( +¥ 2002) 


n>1 
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The sequence of coefficients is sequence A0Q01187 in The Online Encyclopedia of 
Integer Sequences [64]. In the rest of this Chapter, the sequences appearing in the 
Online Encyclopedia of Integer Sequences are referred by their code in that database. 
Using the expansion 
a 
log(l+t)=t-~+7=--::: 
og(1 +1) 7 +3 
and extracting coefficients, it is easy to see that C,, ~ Gy. In other words, almost all 
graphs are connected. But this can be proved more easily using the theory of random 
graphs, as discussed next. 
Consider the random graph binomial model G(n, p) with n vertices, in which 
every edge is drawn independently with probability p. The probability of a graph 
with m edges is 


pm (1—p)@-™. 


If p = 1/2, then every graph has the same probability 2-3), and the distribution is 
uniform among all graphs with n vertices. We say that almost all graphs in G(n, p) 
satisfy property if the probability that a graph with n vertices satisfies </ tends 
to 1, as n tends to -~. It is easy to prove (see [10]) that the following properties hold 
for almost all graphs in G(n, 1/2), hence they hold for almost all graphs under the 
uniform distribution. Let R,, be a random labeled graph with n vertices. Then almost 
surely, as n — 9, R, satisfies: 


e R, has diameter two. In particular, it is connected. 
e R, is k-connected for every fixed k > 1. 
e For every fixed graph H, R,, contains an induced subgraph isomorphic to H. 


e R, is Hamiltonian. In particular, it has a perfect matching if the number of 
vertices is even. 


e R, has no non-trivial automorphisms. 


In the other direction, almost no graph is regular, or planar (since it contains Ks 
as a subgraph), or r-colorable for fixed r (since it contains K,,1 as subgraph). One 
can prove much deeper results using the theory of random graphs. For instance, the 
chromatic number of a random graph with n vertices is close to n/(2log,n), and 
the independence number is close to 2log,n. We will not pursue this topic here, the 
interested reader can find many more results in the reference textbooks [10, 36] of 
this fascinating topic. 

Throughout the paper we use the following conventions. Given a class of labeled 
graphs Y, let G,, = |Y,,| be the number of graphs in Y with n vertices. The associated 
generating function is 


G(x) =P G5. 


As atule we use the same letter for the class, the sequence we want to enumerate and 
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the corresponding generating function. We use the variable x for marking vertices, 
and variable y for edges. If G(x,y) is a series in two variables, G,(x,y) and G,(x,y) 
denote the partial derivatives. We make systematic use of the symbolic method, as 
in [24]. For instance, if Y is a class of graphs closed under the operation of taking 
connected components, the exponential formula 


G(x) = eC) 


relates the generating function G(x) of graphs in Y with C(x), the one for connected 
graphs in Y. As another example, xG'(x) = YnG,x"/n! is the generating function 
of rooted graphs, since a labeled graph with n vertices can be rooted in n different 
ways. 

The notation and terminology for graph theory is standard. Given a graph G = 
(V,E), the subgraph induced by U C V is the graph (U,A), where A is the set of 
edges joining vertices of U. A set U of vertices is independent if there are no edges 
joining vertices of G. A graph is k-partite if the vertex set can be partitioned as 
V =V,U.---UV; such that each V; is independent. The chromatic number 7 (G) is the 
minimum number k of colors in a coloring of V such that adjacent vertices receive 
different colors. A graph G is k-partite if and only if y¥(G) < k. A graph is bipartite 
if and only if it has no odd cycles. 

The rest of the paper is organized as follows. In Section 6.2 we present basic 
decompositions of graphs and the corresponding equations for the associated gen- 
erating functions. This is applied in particular to arbitrary and bipartite graphs. In 
Section 6.3 we discuss the enumeration of connected graphs with given excess; this 
is a classical problem with strong connections to the theory of random graphs. Sec- 
tion 6.4 is devoted to regular graphs, another classical problem. In Section 6.5 we 
cover monotone and hereditary classes, defined in terms of forbidden graphs. This is 
an active area of research closely related to extremal graph theory. In Sections 6.6 
and 6.7 we discuss the enumeration of planar graphs, graphs on surfaces, and classes 
of graphs defined in terms of excluded minors, which are more recent topics of re- 
search. In Section 6.8 we present briefly some results on the enumeration of digraphs. 
We conclude in Section 6.9 with a quick overview on the enumeration of unlabeled 
graphs. 


6.2 Graph decompositions 


The results in this section are based on the classical decompositions of graphs into 
k-connected components, and on the decomposition of a connected graph into the 2- 
core and the trees attached to it. As is well-known, a graph decomposes into its con- 
nected components, and a connected graph decomposes into its 2-connected compo- 
nents, also called blocks: A block is a maximal 2-connected subgraph or a separating 
edge. The decomposition into blocks has a tree structure. Furthermore, a 2-connected 
graph decomposes into its so-called 3-connected components: These are not neces- 
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sarily subgraphs of the host graph and the decomposition is more difficult to describe 
(see for instance [15, 27] for a self-contained discussion). Briefly, one decomposes 
a 2-connected graph G along its 2-separators in a canonical way into bricks, which 
are either cycles, multi-edges or 3-connected graphs. The decomposition is tree-like 
and can be encoded using generating functions as discussed next. 


6.2.1 Graphs with given connectivity 


Let Y be a class of graphs with the following properties: 
(C1) A graph G is in Y if and only if the connected components of G are in Y. 
(C2) A connected graph G is in Y if and only if the blocks of G are in Y. 


This holds clearly for the class of all graphs, but also for other classes such as planar 
graphs. 

The following derivations can be found in [32]. We denote by @ the connected 
graphs in Y, and by & the 2-connected graphs in Y. The corresponding generating 
functions are denoted by 


x” xh x 
G(x) = Gn—, Cx) =e, B(x) = Bn 


The decomposition of a graph into connected components, and the decomposition of 
a connected graph into 2-connected components implies the equations 


G(x) =e, (6.1) 


and my 
C! (x) = eB OC), (6.2) 


Equation (6.1) is the exponential formula for sets of labeled combinatorial objects, 
and (6.2) is based on the recursive decomposition of a rooted connected graph into 
its blocks. The generating function );nC,,x"/n! enumerates connected graphs rooted 
at a vertex, since a labeled graph can be rooted at n different vertices. 
It follows that 
C(x) =logG(x), 


and R(x) =xC’(x) satisfies the implicit equation 
R(x) = xeP (RO), 


We can rephrase the former equation by saying that R(x) is the functional inverse of 
xeB'(x) 

If ¥ is the class of all graphs then G, = 2(3). The former equations determine 
C(x) and G(x) uniquely and we obtain 


x2 x x4 5 


6 
x x 
C(x) Hats a3 387 +7285 426704 +++ 
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which is sequence A001187. Using this expansion we also obtain 


2 3 4 5 6 
X x x 
B(x) =—+—4 107, +2385 + 113685 + ---, 


which is sequence A013922. 
The previous analysis can be enriched by enumerating graphs also by the number 
of edges. Let G,, , be the number of graphs with n vertices and k edges and let 


yx" 
Y= Vr 


and define analogously C(x,y). Notice that G(x,y) is an ordinary generating function 
in y, since edges are not labeled. 
Equations (6.1) and (6.2) generalize to 


G(x, y) — e) ’ Cy (x,y) = xeBr OC (xy) ») ? 


where R(x,y) = xC,(x,y). These equations hold because connected components and 
blocks do not share edges. 
If again & is the class of all graphs then 


own =¥ (O)sn= =ECie 5 


nm \ 
We obtain 
32 ee. 4 
C(x,y) =x+y5, + Gy +y \F + (ly? + 15y* + 6y° +y \F ane 
ag 2 3 x4 
PP caer oe - 
Bx) = Ya tI 3 + (By? +6y? +y ae soe 


Next we turn to the enumeration of 3-connected graphs using the decomposition 
of 2-connected graphs into 3-connected components, for which we follow [66]. We 
add the hypothesis 


(C3) A 2-connected graph G is in Y if and only if the 3-connected components of G 
are in Y. 


Define a network as a 2-connected graph rooted at a directed edge (which may be 
deleted or not) and whose endpoints, called poles, are not labeled. Let D(x, y) be the 
generating functions of networks. Then 


2(1+y)By(x,y) = (1 + D(x,y)). (6.3) 


The left-hand term corresponds to marking and directing an edge, and keeping or not 
the edge; the right-hand side corresponds to adding the empty network and labeling 
the poles. We find 


x2 
D(x,y) =y+ QO? +y°)x+ (29° +757 + 6y? +y ot et 
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Let T (x,y) be the generating function of 3-connected graphs. The decomposition 
into 3-connected components gives rise to the equation 


xD(x,y)? 
Ow 5 EG, (29). 


1+D(sy) =(1-+y}exp (AS? ae 


Given D(x, y), this equation determines T (x,y) and 
cis x” 
T (x,y) = °F + (I5y° + Oy? ty) + 


Setting y = | we obtain the counting series of 3-connected graphs 


x4 x” x6 Pei 
T(x, 1) = Zi +2605 1768 + 22509657 +++: 


This last sequence is A005644. 
As noticed in [28], it is possible to integrate (6.3) with respect to y and express 
B(x,y) as an explicit function of D and B: 


B(x,y) =T(x,D(x,y)) — 5xD(x,y) + log +xD(x,y)) 


= (D6..) ap sDoaye+ (1 +D(x,y)) log (=H) 


It is worth noticing that all the previous equations can be proved more combinatori- 
ally using grammars and the dissymmetry theorem for trees [15]. We summarize the 
previous results in a single statement. 


Theorem 6.2.1 Let Y be aclass of graphs satisfying conditions (C1), (C2) and (C3), 
and let G(x,y),C(x,y),B(x,y),D(x,y) and T (x,y) be the generating functions asso- 
ciated to, respectively, graphs, connected graphs, 2-connected graphs, networks and 
3-connected graphs in Y. Then 


Gy) =e) 


Cr(x,y) = eB OCxy).9) 
¥(1+D(,y)) 


By (x,y) = 2(1+y) ; 
Deny) = (b+ s)exp (PP + 25 (e,Dle))) 1 
B(x,y)  =T(x,D(x,y))- 5xD(x,9) + log(t +xD(x,y)) 


ra (20. ¥ sD)? + (1+D(x,y)) log (pen) | 


For the class of all graphs, starting from the easy expression for G(x, y) we reach 
an equation defining T (x,y). In other situations it is the opposite way; as we will see 
in Section 6.6, for planar graphs one starts from the known expression for T (x,y) and 
then goes all the way up to G(x, y). 


404 Handbook of Enumerative Combinatorics 


Differential equations. In addition to the previous system of equations, it is possi- 
ble to obtain directly B(x, y) and T (x,y) as solutions of second order partial differen- 
tial equations. This can be done algebraically or more combinatorially as in [68] and 
[67]. Setting B = B(x,y) and T = T(x, y), one has 


(2(1+y)By —x°(1+Byx)) (1 —xByx) — 2° BY, = 0 


ang 2 4 2 4.4 
x x"F, x"y 
1 T, == 
(+9) — 7 Te 4F,  4(1+xy)?’ 
where ; 
xy 2 
F =log(1 — —— — —T,. 
og(1 +y) fey 


6.2.2. Graphs with given minimum degree 


In this section we find the generating functions for graphs with minimum degree at 
least two and at least three. For brevity, we define a k-graph as a connected graph 
with minimum degree at least k. 

Given a connected graph G, its 2-core is the maximum subgraph C with minimum 
degree at least two. The 2-core C is obtained from G by repeatedly removing vertices 
of degree one. Conversely, G is obtained by attaching rooted trees at the vertices of 
C. Let H,, be the number of 2-graphs and H(x) the associated generating function. 
Let T(x) be the generating function of rooted trees, that satisfies T(x) = xe"). The 
generating function of unrooted trees (see Section 4.5.1 in this book) is equal to 


The decomposition of a graph into its core and the forest of rooted trees implies the 
following equation: 
C(x) = A(T (x)) +U (x). 


The second summand takes care of trees, which have an empty 2-core. If we change 
variables z = T(x), the inverse function is x = ze~*. This implies 
D2, a8 


Xx x4 x” x6 
A(x) = xe =—+1 +2 + 12 vee 
(x) = C(xe*) —x+ 7 = 3 oF 535 0585 + 


which is sequence A059166. If as before y marks edges, then 


2, 
H(xy) =C(e™,y) —44 2D 


30 4 5 ek 
as (3y" + 6y +y ate 
The kernel of G is obtained by replacing each maximal path of vertices of degree 
two in the 2-core C by a single edge. The kernel has minimum degree at least three, 


and C can be recovered from K by replacing edges with paths. The kernel may have 
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loops and multiple edges, which must be taken into account since we are dealing 
with simple graphs. Also, when replacing loops and multiple edge by paths the same 
graph can be produced several times. This can be dealt with by weighting multigraphs 
appropriately according to the number of loops and edges of each multiplicity. Given 
a multigraph M with a; vertices incident to exactly i loops, and with B; edges of 
multiplicity i, the associated weight is 


wiG)=\] (sa) (1). 


i>l i>] 


The justification is that when replacing an edge of multiplicity i with i different paths, 
the order of the paths is irrelevant, and similarly for the loops. This weight is called 
the compensation factor in the literature [35]. 

We have 0 < w(G) < 1, and moreover w(G) = 1 if and only if G is simple. With 
this definition, the sum of the weights of all 3-multigraphs with n vertices is finite. 
Working first with multigraphs and then moving to simple graphs by setting to 0 the 
variables involving loops and multiedges, one shows the following [47] (this can also 
be deduced from the results in [34]). 

Let K(x,y) be the generating function of simple 3-graphs. Then 


K(x,y) a C (A(x, y), B(x, y)) +E(x,y), 


where 
eer = 


epee = 


Bixy) =(1 sy)exo aie ) 1, 


1+xy 


an xy? xy 


1 
E = ] 1 ? 
(x,y) 2+ 2xy i 5 x 5 n(1+xy) 
In particular 
3s x x x! 
K(x,1) =F +265 + 18585 +2369265 +: 


This sequence is not referenced in OEIS. 
Finally e# () and eX enumerate graphs (not necessarily connected), with mini- 
mum degree at least two and three, respectively. 


6.2.3 Bipartite graphs 


In this section we compute the number of bipartite graphs. The number of bipartite 
graphs with parts of sizes a and b is (eA) 2¢ but in this way a graph may be counted 
more than once. A 2-colored graph is a bipartite graph equipped with a fixed biparti- 
tion. A connected bipartite graph gives rise to two 2-colored graphs, and a bipartite 
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graph with c connected components to 2° such 2-colored graphs. The number g,, of 
2-colored bipartite graphs is 


&n = y @) gh ke 


k=0 


Since each connected bipartite graph is counted exactly twice, the generating func- 
tion Cp(x) for connected bipartite graphs satisfies exp(2Cp(x)) = ¥.gnx"/n!, and 


x2 4 5 


1 x” xe x x 
Co(x) = 5 log (Xe) Sot ea Pao gt ee tet 


2! 


which is sequence A001832. Hence the generating function for all bipartite graphs is 


equal to 
n N Fe 
- n nk) 
Gy (x) = el ) = ( ) (;) 2 ne) 


n k=0 


2 3 4 x 
x x x 
sitar Pie tala + 3765 bak 


which is sequence A047864. The dominant term in the sum for g, is when k = 


n/2, giving gr+m/4 as the exponential growth. As for general graphs, almost every 
bipartite graph is connected. 

A graph is bipartite if and only if all its blocks are bipartite. Hence, according to 
Equation (6.2), the generating function B, (x) of 2-connected bipartite graphs satisfies 


Cx) = ePOG(), 


This gives 
x xe cal xo 

=> +435, +105 + 698657 + ; 
which is sequence A004100. Recurrence relations for the previous numbers are de- 
rived in [33]. As far as we know, the problem of counting 3-connected bipartite 
graphs is open. The difficulty here is to control whether a graph is bipartite in terms 
of its 3-connected components. 

Counting k-partite graphs for k > 2 is not so simple, since a connected k-colored 
graph can be colored in a number of ways that depends on the graph. We discuss this 
problem in Section 6.5. 


By (x) 


6.3 Connected graphs with given excess 


The excess of a connected graph G is defined as |E(G)| — |V(G)|, the number of 
edges minus the number of vertices. Trees have excess —1, and graphs with excess 
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0 are precisely unicyclic graphs, that is, connected graphs with a unique cycle. For 
k > —1, let C,,,4, be the number of connected graphs with n vertices and excess k. 
Computing and estimating C,,,+, 18 a classical problem that arises in the study of 
random graphs [21, 10, 35], and is a crucial ingredient in the analysis of the giant 
component phenomenon. 

The number of trees is C, ,—1 = n"~?, by the well-known Cayley’s formula. There 
are several formulas for the number of unicyclic graphs. Wright [74] showed that 


cnn) = 5 (M22 (ny), 


where 


Hey (") s(n—sy". 


s=1 \S 


Similar expressions were obtained for larger excess. For instance, 


1 
Cunt = 54 ((n—1) (Sn? +3n+2)n"* — 14h(n)). 


The first values for these sequences are as follows: 


n 3. 4 5 6 7 8 OEIS 
Cin 1 15 222 3660 68295 1436568 A006351 
Can+i 6 205 5700 156555 4483360 A058864 


These formulas follow from the general recurrence relation found by Wright: 


2(n+k+1)Cinse+1 = 2(n(n — 1)/2—n—k)Cr nse (6.4) 


n /n k+1 
+ y @ s(n — y) 3 Cs stnCn—s.n—stk—h: 
s=1 


h=—1 


The proof is based on analyzing the result of removing one edge from a connected 
graph, producing either a connected graph or a graph with two connected compo- 
nents. Wright also showed that, for k > 1, 


Crn+k = (—1)'My(n)h(n) + (—1)* (0) (0 1)", (6.5) 


where M; and P; are polynomials in n that can be computed recursively. This implies 
that, for k > 1, the generating function 


x” 
Wi (x) = LCnnte 


is a rational function in T(x) = ¥,51 nrl zr, the generating function of rooted la- 
beled trees, which satisfies the equation T(x) = xe7). 
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The first values of the W;(x) are 


x)? 
W(z) = ; (ve 5 T (x) rw) ) 


T (x)*(6-T(x)) 
24(1—T(x))3 ” 


The expression for W_, (x) is the well-known generating function for unrooted trees, 
as we have already remarked. The one for Wo(x) follows from viewing a unicyclic 
graph as an undirected cycle of rooted trees of length at least three. 

From recurrence (6.5) Wright obtained an estimate for C,, y+, for fixed k. In a sub- 
sequent paper Wright [75] extended this estimate to values k = o(n'/ 3). The estimate 
is 


Wiz) = 


Criyntk = fen Vy (1 + o(n-'/?)) , 


where Jat 
1 13" (k — 1)dy 
fo=g, Sk HS-T/2E(3k/2)' 2 
and the d; satisfy the recurrence 
>  dndy—n 
ne en d d 1. 
1=42= 3e k+l = + pe (e+ 1)(5” 2 


Later on, Bender, Canfield and McKay [2] translated recurrence (6.4) into differential 
equations and extended the estimates to the widest range of parameters. This result 
is reproved by Pittel and Wormald [49] in a different way, using the 2-core (Section 
6.2) of a connected graph. They first estimate the number of connected graphs with 
minimum degree at least two (called 2-graphs in Section 6.2) with given number of 
vertices and edges, and then apply it to estimate the number of connected graphs. 

A purely analytic approach to the problem was developed in [23]. The authors 
find an integral representation for the generating function of connected graphs 


Clay) toe (14 14 Z.), 


n>1 
which linearizes the quadratic exponent in y. They reprove the fact that 
Ax(T(z)) 
Ui) 
where the A, are polynomials, related to the Airy Ai function. For fixed k > 2 they 
obtain complete asymptotic estimates for Cy, y+. It is shown that 


Ar(1)/% 1 at = K {3 1 
_ 4k n_(3k—1)/2 Axl 
Cantk = FG%-12" n / r(#) +B? T o( ) 4 


Wy(z) = 


and lower order terms depend on the derivatives Au ) (1). 
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The values A;(1), which give the first order asymptotics, can be computed di- 
rectly from the following identity 

Y Ax(1)(—x)* = log (: ap a as 

k=1 k= 


=1 


where 
(6k)! 


“k (3k) 1 (2k) 13225k 


The higher derivatives can be computed in terms of the Airy function. 


6.4 Regular graphs 


A graph is d-regular if all the vertices have degree d. Let R,,q be the number of 
(labeled) d-regular graphs with n vertices. Read [56] gave an exact formula for Ryq 
using Pélya’s counting theory, but this formula is not suitable for asymptotic analysis. 
The key to the enumeration of regular graphs is the pairing model. 


6.4.1 The pairing model 


In the pairing model for d-regular graphs, there are n labeled cells, each with d la- 
beled elements, corresponding to the half-edges at each vertex. A pairing of the nd 
elements produces a d-regular graph by regarding the cells as vertices and the pairs 
as edges. Notice that loops and multiple edges may be created by choosing a pair 
from the same cell, or two different pairs joining elements from two distinct cells. 
The number of such pairings is 


(dn)! 
(dn/2)!24"72 


One can estimate the probability that the resulting graph is simple, by considering the 
number X; of cycles of length i created in the pairing. Using the method of moments, 
it can be shown that the random variables X1,...,X, are asymptotically (when n goes 
to infinity) independent Poisson distributed with means A; = (d—1)'/(27) . Hence the 
probability of being simple, which is the probability of not having cycles of lengths 
one or two, is asymptotically ee, This gives 
P(simple) ~ ell-a?)/4, 
Since a simple graph is produced exactly d!” times (permuting the d elements at each 
cell gives the same graph), one obtains for fixed d 


1—d? (dn)! 
4 } (dn/2)!240/2q\n" 


Ria ~ eXp ( 
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Using Stirling’s estimate this can be rewritten as 


n/2 
Rig~ V2el-?)/4 co : dn/2 (6.6) 
n,d é ed(dly2 n : 


The dominating term in the expression above is n’"/2. corrected by an exponential 
term and a constant that depend on d. This estimate was obtained independently by 
several authors (see the historical discussion in [70]). 

For d growing with n, estimating the probability that the resulting graph is simple 
is considerably more difficult. McKay [42] introduced a technique called switchings 
and extended the range of d to d = o(n'/3) by showing 


1-2? @  (@ 
P(simple) ~ exp ( ri Dn” O ( " )) . 
Then McKay and Wormald [43] used a new sort of switching to find the formula for 
d =o0(Vn). 

The basic idea of a switching can be explained through the following example. 
Suppose a pairing has a pair p;p2 that produces a loop in the resulting multigraph. 
Choose another pair p3p4 and replace the two pairs by pip3 and p2p4. The pair 
giving rise to a loop disappears but a new loop can be created. Using double counting 
one can estimate the probability of these types of events, and then obtain an estimate 
of the ratio |S,|/|S,—1|, where S, is the number of parings having exactly a loops. 
By telescoping one can estimate |So|, the number of pairings without loops. More 
general switchings can involve more than two pairs [70]. 

The previous analysis usually extends to graphs with a given degree sequence. 
Let d) > dz >--- > dy, > 1 with Yf_, dj = 2m. Let Gy,a be the number of graphs 
with degree sequence d = (d,...,d,,). One can use again the pairing model with n 
cells containing d,,...,d, elements. The probability that the resulting graph is simple 
can be estimated (for d; bounded) as 


P(simple) ~ ghee 


HO) 


i=l 


where 


When all the d; are equal to d, one recovers the previous expression for d-regular 
graphs. This can be extended in some cases when the d; grow with n. 

The case of 2-regular graphs is particularly simple, since they consist of a col- 
lection of cycles of lengths of at least three. If R(x) is the EGF of 2-regular graphs, 


then ey 
1 1 x2 en 2—x-/4 
R = ] = 6.7 
2(x) = exp 5 (108 x >) AS (6.7) 
3 x4 5 x6 
=3, t3gtlg tit ; 
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which is sequence A001205. Singularity analysis leads immediately (see [24, Exam- 
ple IV.1]) to the estimate for the number of 2-regular graphs 
eo 3/4 
n! 


Jan ” 


which agrees with (6.6) for d = 2. 

Let us mention that the pairing model is the key for analyzing random regular 
graphs. Many properties, like the existence of Hamilton cycles or the value of the 
chromatic number, can be proved using this model [36, 70]. 


6.4.2 Differential equations 


Let Rg(x) =, Rag x be the generating function of labeled d-regular graphs. Gessel 
[25] proved that Ry(x) is D-finite for each d > 2, that is, satisfies a linear differential 
equation with polynomial coefficients. 


Theorem 6.4.1 For each d > 2, the generating function of d-regular graphs is D- 
finite. 


For d = 2, the explicit expression (6.7) immediately gives 
2(1 —x)R}(x) —x?Ro(x) =0. 


For d = 3 it was first proved in [56]. In [69] one finds in addition differential equa- 
tions for the generating functions of connected, 2-connected and 3-connected cubic 
graphs, and for d = 4 it was proved in [58]. In both cases the arguments were com- 
binatorial, based on removing one edge from a d-regular graph and analyzing the 
possible resulting configurations. 

Let R3(x) = Q(x”). Then 


6x7 (2—2x—x°)O" (x) — (x° + 6x4 + 6x3 — 32x +8)0'(x) + = (2—2x—2)°Q(x) =0. 


The first terms are 


10 


4.19355". + 111808202 — + 
8! 10! 


4 6 
x x 
which is sequence A002829. 
A linear second-order differential equation with polynomials coefficients can be 
obtained also for 4-regular graphs but it is a bit long to write down. The initial terms 


are 
6 7 8 


x” x x x 
Ra(x) = 51 +156 465-5, 19355 sownds 


which is sequence A0058 15. 

The problem was revisited by Goulden, Jackson and Reilly in [30], in terms of 
symmetric functions. Consider labeled graphs on vertices {1,2,...,} and let t; be a 
variable marking the degree of vertex i > 1. Then 


= I] (1 + tt) 


1<i<j 
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is the generating function of labeled graphs marking all possible degrees, because if 
{i,j} is an edge, there is a contribution of one to both the degree of i and j. This is 
a symmetric function in infinitely many variables t = (t),f2,...). The number of d- 
regular graphs on n vertices is the coefficient [r¢ ---14]T (r). These coefficients (called 
regular in [30]) can be expressed in terms of the so-called H-series of T. The authors 
then showed that the H-series satisfies a system of partial differential equations. From 
here they proved D-finiteness of Ry(x) for d = 2,3,4, but the general case remained 
open. 

Gessel [25] proved D-finiteness in the general case and at the same time extended 
it to graphs with loops (a loop contributes 2 to the degree of a vertex) and multiple 
edges, by considering variations of the T symmetric function. For instance 


I] (1 +tit;) 


1<i<j 
corresponds to graphs where loops are allowed (by taking i = /), and 


1 
1 —t;t; 


1<i<j 


to graphs where loops and multiple edges are allowed. 


6.5 Monotone and hereditary classes 


In this section we discuss classes of graphs closed under the operation of taking sub- 
graphs or under taking induced subgraphs. This is a very active area of research with 
close connections to extremal graph theory and related areas. Most of the results are 
asymptotic and only in a few cases does one have access to the counting generating 
functions. 


6.5.1 Monotone classes 


A class of graphs Y is monotone if it is closed under taking subgraphs. Every mono- 
tone class is defined by a collection ¥, possibly infinite, of minimal forbidden sub- 
graphs. These are the graphs G not in Y but such that every proper subgraph is in Y. 
We denote by Mon(#) the class of graphs not containing any subgraph isomorphic 
to a graph in #. Next we briefly describe the relation between monotone classes and 
extremal graph theory. 

Denote by ex(n;F’) the maximum number of edges in a graph with n vertices 
not containing F as a subgraph. More generally, ex(n;-¥) is the maximum number 
of edges in a graph with n vertices belonging to Mon(.¥), that is, containing no 
subgraph in #. Let G be a graph in Mon(#), with maximal number of edges. Since 
every subgraph of G is also in Mon(.¥), we have 


|Mon(F),| > 2%"). 
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It turns out that in many cases this lower bound is not too far from the right an- 
swer. This is a consequence of the fundamental Erdés-Stone-Simonovits Theorem in 
extremal graph theory. Let 7(G) be the chromatic number of a graph G, and let 


r=min{y(F): Fe F}—-1. 
Then the extremal function ex(n; Mon(.¥)) satisfies 


1 n 


ex(n;Mon(.¥)) = (1 = ~) (5) +o(n?), asn—o., 


For r = | this result says that the number of edges is subquadratic, and for r > 2 it is 
quadratic and of order (1 —1/r)n?/2. 

A fundamental result of Erdés, Frankl and R6édl [19] gives the rough order of 
magnitude for the number of graphs in a monotone class, when r > 2. 


Theorem 6.5.1 Let Mon(.7) be a monotone class with r > 2. Then 
|Mon(¥)n| = 20-7 +0) G), 


Notice that the o(1) term is in the exponent, hence this estimate is very far from being 
precise. When r = | (at least one bipartite graph is forbidden) the situation is more 
complicated. A case that has been much studied is Mon(Cy4). It is proved in [37] that 
the number F,, of graphs not containing C4 satisfies 


3/2 at 3/2 
gor? <p gen? 0cex<c, 


an estimate that is still far from being precise. 

In some cases though one has very precise estimates. We start discussing the 
class of triangle-free graphs. Clearly every bipartite-graph is triangle-free. It was 
proved by Erdés, Kleitman, and Rothschild [20] that, somewhat surprisingly, almost 
all triangle-free graphs are bipartite. Let B,, be the number of bipartite graphs. 


Theorem 6.5.2 Almost all triangle-free graphs are bipartite. More precisely, 


|Mon(K3)| = Bn (1 +0 @) 


Since the number of bipartite graphs is known (see Section 6.2), this gives a pre- 
cise estimate for the number of triangle-free graphs. This fundamental result was 
extended by Kolaitis, Promel and Rothschild [38]. 


Theorem 6.5.3 Almost all K;41-free graphs are t-partite. 


The Erdés-Kleitman-Rothschild theorem was generalized in a different direction, by 
showing that almost all graphs in Mon(C2+1) are bipartite [39]. The two previous 
results were widely generalized by Promel and Steger [53]. An edge e of a graph H 
is color-critical if y¥(H — e) < 7(H). Notice that both in complete graphs and odd 
cycles every edge is color-critical. 
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Theorem 6.5.4 Let H be a graph containing a color-critical edge that satisfies 
4(H) =r+1 > 3. Then almost all graphs not containing H as a subgraph are r- 
partite. 


In fact the authors show that the previous condition is also necessary: If almost every 
graph in Mon(H) is r-partite then H contains a color-critical edge. 

We now discuss the problem of counting k-partite graphs. A simpler problem is 
to count k-colored graphs. A k-colored graph is a graph together with a particular k- 
coloring using all k colors. A recursion for the number c, , of k-colored graphs was 


obtained in [57]: 
n n ye 
ties). 6 Deora, 
j=0 


with the initial conditions co, = 1 and c,9 = 0 for n > 1. Prémel and Steger [55] 
showed that almost all k-partite graphs are uniquely k-colorable, hence the number 
of k-partite graphs is asymptotically k!c, %. Estimates for c,, are given in [71]. The 
dominant term is 
1) n2 
20 -£)"5 pn 


and the subexponential term depends on a subtle way on the value of nm modulo k. 


6.5.2 Hereditary classes 


Recall that a class of graphs is hereditary if it is closed under the operation of taking 
induced subgraphs. Notable examples include perfect graphs, and several subclasses 
such as chordal graphs, interval graphs or permutation graphs [29]. A graph G is 
perfect if for every induced subgraph H, y(H) equals the size of the largest complete 
subgraph in H. A graph is chordal if every cycle of length more than three contains a 
chord. The theory of perfect graphs is closely related to combinatorial optimization 
[29]. 

A hereditary class Y is characterized by the list, not necessarily finite, of for- 
bidden subgraphs, the minimal graphs not in Y all whose proper induced subgraphs 
are in Y. We write Forb(H) for the class of graphs not containing H as an induced 
subgraph. The class Mon(K;) discussed previously is hereditary, since a K; subgraph 
is always induced. However the class Mon(Cy4) is not hereditary, since a graph can 
contain copies of C4, none of them induced. 

A general asymptotic result is available for hereditary classes, similar to Theorem 
6.5.1. It is based on a modified version of the extremal function ex(n,H) that makes 
sense for induced subgraphs. The following concept was introduced in [54]. Given 
positive integers s <r, an (r,s)-coloring of a graph G is a coloring c: V(G) = [r] 
such that vertices with color i induce a clique for 1 <i <_s, and an independent set 
for s-+1<i<r. Notice that (7,0)-colorable is the same as r-colorable in the classical 
sense. The coloring number of a hereditary class Y is defined as 


r(Y) =max{r: there exists s <r such every (r,s)-colorable graph is in G}. 


The following was proved in [54] for hereditary classes excluding a single graph, and 
in the general case in [12]. 
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Theorem 6.5.5 Let Y be anon-trivial hereditary class of graphs with coloring num- 
berr =r(Y). Then 
IG| = 2-F +e) G). 


Observe that again the o(1) term in the exponent makes the estimate very far from 
being precise. 

In a different direction, it is know that the rate of growth of the number of graphs 
in a hereditary class cannot be arbitrary. This was first established in [61], and later 
extended by several authors [11]. In the next statement B() is the Bell number (the 
number of partitions of an n-set), that grows like B(n) ~ ((1 +0(1))n/logn)". 


Theorem 6.5.6 Let Y be anon-trivial hereditary class of graphs with coloring num- 
ber r = r(Y). Then one of the following cases holds for sufficiently large n. 


I. |G,| is identically zero, one or two. 
2. |Y,| is a polynomial in n. 


3. |G,| has exponential order of the form ea pi(n)i" for some k > 0 and poly- 
nomials pj. 


4. |Y,|=n0-"/r+0)) for r> 1. 
5. B(n) <|F| <2. 
6. |G| = 2AM rte) /2 for ¢ > 1. 


These results extend to other combinatorial structures such as oriented graphs, hy- 
pergraphs, or posets. 

Next we turn to particular hereditary classes. A fundamental difference between 
Mon(Cy4) and Forb(C,) can be observed by looking at the extremal functions. It is 
well-known that ex(n;C,) = @(n?/?), but a graph in Forb(Cy) can have @(n?) edges: 
consider the disjoint union of K,,/. and the complement of K;,/2. The former is an 
example of a split graph. A graph is split if its vertex set can be partitioned into a 
clique and an independent set. Clearly a split graph is in Forb(Cy). The following 
was proved by Promel and Steger [51]. 


Theorem 6.5.7 Almost all graphs in Forb(C4) are split. 


It follows that the number of graphs in Forb(C4) is of the same order as the number 


of bipartite graphs. 
A graph G is generalized split if either G or its complement satisfy the following 
property: the vertex set can be partitioned into disjoint cliques Ho, H),...,H, such 


that there is no edge between H; and H; fori > j > 1. It is easy to check that gener- 
alized split graphs contain no induced Cs. The following was proved in [52]. 


Theorem 6.5.8 Almost all graphs in Forb(Cs) are generalized split. 
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The former result has a striking consequence. A graph G is perfect if @(H) = 
~(H) for every induced subgraph H of G, where @(G) is the size of the largest 
completed subgraph in G. A perfect graph does not contain an induced Cs and it can 
be checked that generalized split graphs are perfect. Hence we have 


Generalized split C Perfect C Forb(Cs). 
It follows that 
Theorem 6.5.9 Almost all perfect graphs are generalized split. 


This is an unexpected result. Perfect graphs are fundamental in graph theory and 
some difficult conjectures have been proved only recently. Let us mention the per- 
fect graph conjecture, which says that perfect graphs are precisely those excluding 
odd cycles of length at least five and their complements as induced subgraphs [16]. 
However, most perfect graphs have a very simple structure. 


Cographs. In the last part of this section we analyze hereditary classes that can be 
enumerated exactly using generating functions. A cograph is a graph not contain- 
ing the path P, as an induced subgraph. They can be characterized alternatively as 
follows: 


e The empty graph is a cograph and a single vertex is a cograph. 


e If Gand H are cographs so are the disjoint union GUA and the sum G+ H 
(the sum is the disjoint union plus all edges between G and H). 


Cographs are relevant in computer science, since several hard computational prob- 
lems can be solved in polynomial time when the input is a cograph [29]. 

Let G(x) and C(x) be, respectively, the generating functions of cographs and 
connected cographs. Since a cograph is connected if and only if its complement is 
disconnected, we have G(x) — 1 +x = 2C(x). Together with G(x) = e©), this gives 


e€) —2C(x) +x-1=0. 


It follows that the dominant singularity of C(x) is at p = 2log(2) — 1. From here it is 
easy to deduce the estimates 


3/25"! = Gy Cy. 


Cy, ~ cn 


p 


This derivation can be found in several places, for instance [31]. 
Two subclasses of cographs that have been considered in the literature are the 
following. 


1. A graph is trivially perfect if it does not contain Py or C4 as induced sub- 
graphs. A similar argument as before gives the equation 


(1—e7*)e© — C(x) =0. 
The dominant singularity is p = 1 —log(e— 1) and 


Cr~en onl, = Ga ~ Cn. 


p 
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2. A graph is a threshold graph if it does not contain Py,C4 or Kz U Kz (two 
disjoint edges) as induced subgraphs. As for cographs we have G, = 2C,. In 
this case we have an explicit expression for C(x): 


1-x 
2-—e° 


C(x) = 


The function C(x) has a simple pole at log(2) and 


Cy ~ clog(2)~"n!, Gn ~ 2Cy. 


The first values for the number of cographs and their subclasses are given in the 
next table. 


n 12 3 4 #5 6 OEIS 

Cographs 1 2 8 52 472 5504 A006351 
Trivially perfect| 1 2 8 49 402 4144 A058864 
Threshold 1 2 8 46 332 2874 A005840 


6.6 Planar graphs 


The enumeration of planar graphs started with Tutte in the 1960s, with his funda- 
mental papers on map enumeration, where a map is a graph with a fixed embedding 
in the plane. Map enumeration has grown into an important area of research; see 
Chapter 5. In this section we are interested in enumerating planar graphs as combi- 
natorial structures, without considering a particular embedding. As we are going to 
see, the enumeration of planar graphs is based on the enumeration of planar maps and 
the graph decompositions described in Section 6.2. Throughout this section, T (x,y), 
D(x,y), B(x,y), C(x, y) and G(x,y) have the same meaning as in Section 6.2 restricted 
to planar graphs. 

The starting point of the analysis is Whitney’s theorem, claiming that a 3- 
connected planar graph has a unique embedding in the sphere up to homeomorphism. 
Let M,,, be the number of rooted 3-connected planar maps with n vertices and k 
edges, and let 7,,; be the number of 3-connected planar graphs with n vertices and k 
edges. Then we have the relation 


M,, xn! = AKT x. 


This is because there are n! ways to label a rooted map with n vertices (since vertices 
in a rooted map are all distinguishable), and there are 4k ways of rooting a planar 
graph with k edges to obtain a rooted map (2k choices for a directed edge and two 
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choices for the root face). Let the generating functions associated to 3-connected 
maps and graphs, respectively, be 


x” 
M(x,y)=YMnay'x", Ty) = Tray". 
The former relation implies 
M(x,y) = 4yTy (x,y). 
This was first made explicit in [4], using the fact (proved in [44]) that M(x,y) is 
algebraic of degree four and is given by 
1 ” 1 (1+u)*(1+v)? 
Itxy I1+y (1+U+V)> }’ 


M(xy) =2? ( 


where 
U=xy(1+v)?, V=y(1+u). 


From the previous equations one can estimate easily the number 7, of 3-connected 
planar graphs as 
Th~ c3n 2 val, 


where ¥3 = (272 + 112\/7)/27 = 21.05. 

The series T(x,y) determines the series B(x,y) of 2-connected planar graphs via 
Theorem 6.2.1. From here, Bender, Gao and Wormald [4] obtained the asymptotic 
number B,, of 2-connected planar graphs as 


Baw con /?nl, 


where 7 ~ 26.18. This was an important step, since it opened the way to the full 
enumeration of planar graphs. 

In order to proceed further one needs an expression for B(x, y) in terms of D(x, y). 
This was achieved by Giménez and Noy [26]. 


Lemma 6.6.1 Let D = D(x,y) be the generating function of planar networks and let 
B(x,y) be that of 2-connected planar graphs. Let W = z(1+U), where U is defined 
in (6.6). Then 


x Xx 
B(x,y) = 5 Fi (x,y) = qB2(,y), (6.8) 
where 
D(6x—2+.xD) l+y log(1+D)  log(1+xD) 
Bi. 2 ee et ; 
4x oe yioe (75) 2 De 
2(14+x)(1+W)(D+W7)+3(W-D) 1 ; 
By = log(1+xD 
2 2 +W! 5: og(1+xD+xW +xW~) 
m 14x (1 wy eee 1—x+xD+—xW +xW? 
Be; 4x (l—x)(D+W2+1+W) J 
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The former expression is obtained by integrating T,(x,y) with respect to y, and using 
Equations (6.3-6.2.1). It is remarkable that the same result can be recovered in a 
purely combinatorial way using grammars [15]. 

With this explicit expression one can check that the equation 


Cc’ (x) = eB 0C'(x)) 


has no branch point. This is equivalent to the fact that B’(R) < 1/R. It follows that 
the singularity p of C(x) is given by p = Re~®'®), where R = 1/7, and is of the 
same kind as that of B(x). Finally, the singularity of G(x) = eC”) is the same as that 
of C(x). Applying singularity analysis one obtains estimates for the numbers C,, and 
G, of the same kind as for 7, and B,. The results are summarized as follows. 


Theorem 6.6.2 For £ € {0,1,2,3}, the numbers of ¢-connected planar graphs are 
asymptotically of the form 

con 7! win! 
where 


WwY21.05, p26.18, Y%=Y%~27.23. 


Moreover, the asymptotic probability that a random planar graph is connected is 


c1/Co = 0.95. 


We include a table with the numbers of small planar graphs of given connectivity, 
with the convention that a k-connected graph must have at least k + 1 vertices. 


n 123 4 5 6 7 OEIS 

Planar 1 2 8 64 1023 32071 1823707 A066537 
Connected 1 4 38 727 26013 1597690 A096332 
2-connected 1 10 237 10707 774924 A096331 


3-connected 1 25 1227 84672 A096330 


Itis also possible to estimate the number of planar graphs with given edge density. 


Theorem 6.6.3 For each @ € (1,3), the number of planar graphs with n vertices and 
|an| edges is asymptotically 


-4 


c(a)n-“*y(a)"n!}, (6.9) 


where y(t) is an analytic function that achieves its maximum at 0 = kK = 2.21. The 
expected number of edges in a random planar graph is precisely kn. 


The proof is based on a local limit theorem. For each @ € (1,3) there is a y= y(@) 
such that giving a graph with k edges the weight y“, the weight is concentrated on 
graphs with an + O(,/n) edges. For this value of y, the generating function G(x, y) 
has radius of convergence p (a) and we have 


'G,y)~ enya), (a) =p(a)!. 
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The extra n~!/? factor in (6.9) comes from the local limit theorem (for details, see 
[26]). 

The previous enumerative results opened the way to the fine analysis of random 
planar graphs. Basic parameters such as the number of edges and the number of 
connected components were already analyzed in [26]. Extremal parameters, such as 
the size of the largest component or the size of the largest block, the maximum degree 
and the diameter can be analyzed too [46]. 

Some special classes of planar graphs have been enumerated too, in particular 
cubic planar graphs [9]. The decomposition into 3-connected components can be 
adapted for cubic graphs. If G is a 3-connected planar graph, its dual (which is unique 
by Whitney’s theorem) is a 3-connected triangulation. Using Tutte’s enumeration 
of triangulations [65] one can express the generating function of 3-connected cubic 
planar graphs in terms of the (ordinary) generating function T(z) of triangulations, 
given by 

T(z) = u(z)(1 — 2u(z)), u(z)(1 —u(z))? =z. 


Theorem 6.2.1 can be adapted to this situation and one can show [9] that the number 
of cubic planar graphs is asymptotically, for even n — 9, 


cn /76"n!, = 6 & 3.13, c>0. 


On the other hand, counting 4-regular planar graphs is yet an open problem. No 
decomposition into 3-connected components is known for 4-regular graphs, and the 
enumeration of general 4-regular graphs [58] does not seem apply when restricted 
to planar graphs. Another interesting open problem is to enumerate bipartite planar 
graphs. 


6.7 Graphs on surfaces and graph minors 


Planar graphs are graphs embeddable on the sphere, hence it is only natural to con- 
sider graphs in other surfaces. Graphs on surfaces are a special case of classes of 
graphs defined in terms of excluded minors. 


6.7.1. Graphs on surfaces 


The analysis from the previous section can be extended to graphs embeddable in a 
fixed surface. The starting point is the enumeration of maps on surfaces. After the 
seminal work of Tutte, the theory of map enumeration was extended to arbitrary 
closed surfaces. Using Tutte’s technique of removing the root edge and using induc- 
tion on the genus, Bender and Canfield [1] proved the following fundamental result 
(see Section 5.4.3 in the previous chapter for a discussion). 
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Theorem 6.7.1 Let M,, be the number of rooted maps of orientable genus g > 0 
with n edges, and let Nyy be the analogous quantity for non-orientable genus h > 1. 
Then 

Men ~ ego Nie i2" 


and 
Nan ma E,nrih—2)/4 12” 


for some positive constants Cg,Ch. 


As for planar graphs, it took some time to go from enumeration of maps to graphs 
in a surface. The corresponding result for graphs was obtained independently in [3, 
14]. 


Theorem 6.7.2 Let G,, be the number of rooted maps of orientable genus g > 0 
with n edges, and let Hj, , be the analogous quantity for non-orientable genus h > 1. 


Gyn wv dg 8 VP2-Vytn! 


and is 
gn ay dgn?(t-2)/4-lytn) 


for some positive constants Cg,c),, and where Y © 27.23 is as in Theorem 6.6.2. 


The number of graphs that can be embedded in the orientable surface of genus g 
has the same estimate, since graphs of genus less than g are of a smaller order of 
magnitude than graphs of genus g. The same applies to the non-orientable case. 

The proof of Theorem 6.7.2 is rather technical, here we limit ourselves to give 
the main ideas. Suppose one wishes, as for planar graphs, to use the enumeration of 
maps on the surface S for counting graphs (without an embedding) on S. There are 
two main obstacles for this program: 


1. No degree of connectivity guarantees a unique embedding. 


2. The class of graphs embeddable in S is not closed under taking connected 
components and blocks (genus is additive on components and blocks). 


Hence the basic equations among generating functions from Theorem 6.2.1 no longer 
hold. The road to the solution is to consider a parameter called face-width. The face- 
width fw(M) of a graph G embedded in S is the minimum number of intersections 
of G with a simple non-contractible curve C on S. It is easy to see that this mini- 
mum is achieved when C meets G only at vertices. Face-width is in some sense a 
measure of local planarity, if the face-width is large then the embedding is planar 
in balls of large radius. The face-width of a graph G is the maximum face-width 
among all the embeddings of G. The key result is that a 3-connected graph with large 
enough face-width has a unique embedding. It turns out that the generating series 
of 3-connected graphs of any fixed face-width has a negligible contribution in the 
asymptotic analysis. Therefore, the enumeration of 3-connected graphs in S can be 
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reduced, up to negligible terms, to the enumeration of 3-connected maps in S. It is 
important to remark that, since maps with small face-width are discarded, one does 
not work with exact counting series. Instead, if f(x) is the series of interest, one finds 
computable series f1(x) and f2(x) such that fi(x) < f(x) x fo(x) (where = means 
coefficient-wise inequality) and fi(x) and fo(x) have the same leading asymptotic 
estimates. 

For the second obstacle one can use a result from [59]: If a connected graph G 
of genus g has face-width at least two, then G has a unique block of genus g and 
the remaining blocks are planar. A similar result holds for 2-connected graphs and 
3-connected components. Since for planar graphs we have exact expressions for all 
the generating functions involved, starting from the (asymptotic) enumeration of 3- 
connected graphs of genus g we can achieve the enumeration of all graphs of genus 
g. To be more precise, let GS (x) and C8 (x) be the generating functions of graphs and 
connected graphs of genus g, respectively. The usual relation G8 (x) = expC (x) does 
not hold, since the union of graphs of genus g can have larger genus. Instead, we have 


G8 (x) ~ C8 (x)eC) 


where the symbol must be understood as that the two functions have the same domi- 
nant asymptotic terms. Similarly, the relation between C$ (x) and the generating func- 
tion B’ (x) of 2-connected graphs of genus g is not an exact equation as in the planar 
case, since genus is also additive in blocks, but rather an approximate version. The 
technical details are quite involved but the essence is to discard maps and graphs with 
small face-width. 


6.7.2. Graph minors 


Graphs on surfaces are one of the main examples of the theory of graph minors. A 
graph H is a minor of G if H can be obtained from a subgraph of G by contracting 
edges. Equivalently, if H can be obtained from G by edge contractions and deletions, 
and deletion of isolated vertices. A class ¥ of graphs is minor-closed if whenever 
G is in Y and H is a minor of G, H is also in Y. The class is proper if it does not 
contain all graphs. The class Ys of graphs that can be embedded in a fixed surface S is 
minor-closed. Other examples are graphs with bounded tree-width and AY-reducible 
graphs. 

Given a minor-closed class ¥, an excluded minor for ¥ is a graph H such that 
H is not in Y but every proper minor of H is in Y. For instance, Kuratowski’s theo- 
rem says that Ks and K3 3 are the excluded minors for planar graphs. The theory of 
graph minors is one of the main achievements in modern combinatorics, culminating 
with the great theorem of Robertson and Seymour: For each minor-closed class the 
number of excluded minors is finite. We write Y = ex(H),...,H,) if the Hj are the 
excluded minors of Y. A class ¥ is said to be decomposable if it satisfies condi- 
tion (C1) in Section 6.2. It is addable if it is decomposable and in addition satisfies 
condition (C2). It is easy to see that a class is decomposable if and only if its ex- 
cluded minors are connected, and is addable if and only if its excluded minors are 
2-connected. 
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Let Y be a proper minor-closed class of (labeled) graphs. We say that Y is small 
if there exists a constant c > 0 such that 


IG,|<c"n!. 


Observe that this is equivalent to the fact that the exponential generating function 
¥|G,|x"/n! has positive radius of convergence. A general enumerative result says 
that a minor-closed class is small. It was first proved in [45] and later in a more 
general context in [18]. 


Theorem 6.7.3 If G is a proper minor-closed class of graphs, then G is small. 


Hence the possible growth rates are bounded by c”n!. A classification of the possible 
growth rates of minor-closed classes of graphs can be found in [7], similar to that for 
hereditary classes in Section 6.5, although growth rates above c”n! are excluded. 

To measure the size of a class we consider y = limsup,,_,.. (G,/nt)!/". Because 
of the previous theorem, if Y is proper minor-closed, then y < ++co. We say that Y 
has a growth constant if the lim sup is actually a limit, that is, 


1/n 
y= lim (=) F 
no \ n! 


The class Y is smooth if the limit lim,_,.. Gy /(nGn_1) exists. This is a stronger con- 
dition than having a growth constant and if this is the case then 


. n 
lim 
no nNGy_| 


The following very general theorem was proved by McDiarmid [41]. 
Theorem 6.7.4 A Y proper addable minor-closed class of graphs is smooth. 


It is an open problem whether every minor-closed class has a growth constant. 

Let I be the set of real numbers that are growth constants of minor-closed classes 
of graphs. It is known [6] that 0,1,2 belong to I’, and the only other number in 
Tn (0,2] is € 1.76, which is the growth constant of forests of caterpillars. Given 
a minor-closed class Y, one shows that either Y contains all paths and its growth 
constant is at least 1, or else it excludes some path and the growth constant is 0. The 
proof that & is the only growth constant in (1,2) uses a similar idea using caterpillars 
instead of paths. Another result is that if y¢ I, then 2y € T. All possible values in 
I'M [2,2.25] are also determined [6], where it is shown that there are infinitely many 


gaps. 


6.7.3 Particular classes 


There are no general results on the enumeration of minor-closed classes besides those 
discussed above. But there are precise results for a number of relevant classes. All 
the following examples satisfy conditions (C1) and (C2) from Section 6.2 (they are 
addable) and the equations in Theorem 6.2.1 apply. The series G,C,B,D,T has the 
same meaning as in Section 6.2. We use throughout the equations relating these gen- 
erating functions stated in Theorem 6.2.1. 
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Forests. A forest is an acyclic graph, that is, all its components are trees. Alter- 
natively, the class of forest is ex(K3). The generating function of trees is C(x) = 
T(x) —T(x)*, where T(x) = xe7“) (see Section 6.3). The generating function of 
forests is 

G(x) =e), 
From here it follows easily that the number of forests is asymptotically (a result first 
proved by Takacs [63]) 


1/2 
e 
ell2nh? w ( ) yoltanl. 
20 


Outerplanar graphs. A graph is outerplanar if it can be embedded in the plane 
so that all the vertices are in the outer face. They are also the same as the class 
ex(K4, K3.2). A 2-connected outerplanar graph has a unique Hamilton cycle and it can 
be drawn in the plane as a polygon plus some non-intersecting diagonals; in classical 
enumeration these are known as polygon dissections [22], and are enumerated by the 
small Schrdder numbers. From here it follows easily [8] that the generating function 
B(x) of 2-connected outerplanar graphs satisfies 


: 14+5x—V1—6x4+.2? 
B(x) = ; 

8 
The singularity of B(x) is at R = 3 — 2/2. One checks that B’(R) > 1/R, which im- 
plies that in this case Equation (6.2) has a branch point, as opposed to what happened 
for planar graphs. In other words, the singularity of C(X) from a branch point and 


not from the singularity of B(x). From here one gets the estimate for the number of 
outerplanar graphs with n vertices 


cn /?y¥'nl, ye 7.32. 


Here and in the sequel c is a computable positive constant, that depends only on the 
class under consideration. 

If we compare with the numeration of forests, we see the same behavior. For 
forests B(x) =x? /2, while for outerplanar graphs it is an algebraic function. However 
in both cases the singularity comes from the existence of a branch point and the 
behavior of C’(x) near its singularity is in both cases of square-root type. This remark 
applies to the next example too. 


Series-parallel graphs. A graph is series-parallel if it does not contain K4 as a 
minor, that is, series-parallel graphs constitute the class ex(K,). Equivalently, series- 
parallel graphs are those that can be obtained from a forest by repeatedly applying 
series and parallel operations. Clearly, outerplanar graphs are series-parallel. The 
starting point for the enumeration is the observation that a series-parallel graph al- 
ways has a vertex of degree at most two, hence it cannot be 3-connected. Hence the 
generating function of 3-connected graphs is identically zero. The equation for D in 
Theorem 6.2.1 reduces to 


xD(x,yP ) 


1+D(x,y) = (1+y) exp (2S z 
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Applying (6.2.1), this gives a direct expression for B(x, y) in terms of D(x, y). It is not 
as explicit as for outerplanar graphs, but the situation is very similar. Equation (6.2) 
has again a branch point and one gets the estimate for the number of series parallel 
graphs 

cn>/?y"n!, —-y 9.07. 


Classes defined by 3-connected components. One can define a class of graphs 
¢ by fixing the class Y of 3-connected members in the class, and letting Y be the 
graphs whose 3-connected components are in 7. For instance, if 7 =, then ¥Y is 
the class of series-parallel graphs. A detailed analysis of classes defined in terms of 
3-connected components is done in [28]. It is shown that the asymptotic enumeration 
of Y depends essentially on the analytic properties of the generating function T (x,y) 
of 3-connected graphs at its singularity (for each fixed value of y). In a number of 
cases one shows that the behavior is similar to the class of series-parallel graphs, with 
a subexponential term n—>/2, In other cases, Y is close to the class of planar graphs 
and the subexponential term is n—'/?. This difference in the asymptotic estimates 
implies a profound difference in the properties of random graphs from Y with respect 
to the size of the largest block [28]. 
We reproduce part of a table from [28] for several minor-closed classes. 


Class a Y 
Series-parallel | —5/2 | 9.07 
ex(W4) —5/2 | 11.54 
ex(Ws) —5/2 | 14.67 
ex(Ks ) —5/2 | 15.65 
ex(K3 x Ky) | —5/2 | 14.67 
Planar —7/2 | 27.2269 
ex(K3,3) ~7/2 | 27.2293 
ex(Ky) ~7/2 | 27.2295 


The constants y and @ are such that the asymptotic estimate is of the form 
cn*y'nh. 


The classes are defined in terms of excluded minors. W,, is the wheel with n vertices, 
K, is the graph obtained from Ks by removing one edge, and Ky. 3 is obtained from 
K3 3 by adding one edge. 

The first five classes in the table are subcritical classes. A class is subcritical if 
Equation (6.2) relating the generating functions B(x) and C(x) has a branch point. 
Hence the singularity of C(x) comes from this branch point and not from the singu- 
larity of C(x). As shown in [17], for subcritical classes the exponent is always —5/2. 
One may notice that in these cases the excluded minors are planar, as opposed to the 
cases where the exponent is —7/2. It is conjectured in [46] that the class ex(H) is 
subcritical precisely when H is planar. 
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All the previous examples are from addable classes. A first investigation of non- 
addable classes is carried out in [13]. Here we list some examples. 


e The class of graphs that contain at most one cycle. These are trees and uni- 
cyclic graphs. The estimate for the number of graphs is 


1 
(2e) "40 (1/4) 


cn /4e"n!, c= 


The unusual exponent comes from a singularity of type fourth-root. 


e The class of forests of caterpillars, that is, all the components are caterpillars. 
The estimate in this case is 


cn 3/4e2V anny 


. a if _ 1-p)? 
where y¥ is the inverse of the only positive root p of xe* = 1,a = At)’ and 
c is expressed in terms of p and y. The unusual asymptotics are due to the 


fact that C(x) has a simple pole, so that G(x) has an essential singularity. 


e The class of graphs all whose connected components have size at most k. The 
generating function for connected graphs is C(x) = Y*_, ¢;x'/i!, where c; is 
the number of connected graphs with i vertices. Hence G(x) = expC(x) is 
the exponential of a polynomial. Applying Hayman’s admissibility theorem, 
one obtains the estimate 

1 Ag), 


ni, 
Vimkn 

where € = ¢,, is the solution to the saddle-point equation €C’(C€) =n, and 

satisfies € = an'/* + B + O(n—!/*) for suitable constants @ and B. 


6.8 Digraphs 


A digraph is a labeled directed graph without loops or multiple arcs, but possibly 
with both arcs uv and vu for a given pair of vertices. The number of digraphs on n 
vertices is of course 2”"—!), since there are n(n — 1) possible arcs joining distinct 
vertices. A source in a digraph is a vertex of indegree zero. A digraph is acyclic if it 
has no directed cycle, and it is strongly connected if any two vertices are joined by a 
directed path. 


6.8.1 Acyclic digraphs 


A very classical problem is to enumerate acyclic digraphs; they are clearly loopless 
and given two distinct vertices u and v, only one of the two arcs uv and vu can be 


Graphs 427 


present. Let a, be the number of acyclic digraphs on n labeled vertices, and let a, be 
the number of those having m arcs. There is a simple recurrence for computing these 
numbers, found independently in [60, 62] and rediscovered several times. The num- 
ber of acyclic digraphs such that k given vertices are sources is equal to gte-Da 5, 
since the remaining n — k vertices induce an acyclic digraph and every arc from the 
sources to the complement is allowed. Given that an acyclic digraph always has at 
least one source, inclusion-exclusion gives 


ne] n : 
an = )B (") (-1)" DN a dao = 1. 
t=0 
This gives the sequence 
1,1,3,19,219,4231, 130023,..., 
which is sequence A001035. More generally, if 


(3) 
An(y) = 2 any” 


is the edge-enumerator of acyclic digraphs with n vertices, then 


n—1 
n 
An(y) = be 8 (— pe (1 HEA; (y). 
t=0 
In order to determine the asymptotic behavior of the a,, we introduce the gener- 
ating functions 


co 


yx" - n e 
A(x) _ oy (3) ? A(x,y) = 3 n : 
n=0 2\2/n! 


Define also 


ea re 


W(x) = ee, 
®) n=0 (+y)Qn! 


The previous recurrence relations imply the equations 


: Al) = gay" 


By locating the smallest zero of ‘¥(x) one obtains the estimate 
an ~ ay'nt2(), 


where y = 0.67201 and A ~ 1.74106. 
The asymptotic enumeration of dense acyclic digraphs (with a quadratic number 
of arcs) is analyzed in [5]. 
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6.8.2 Strongly connected digraphs 


It is well-known that almost all digraphs are strongly connected, since with high 
probability there is a path of length two between any two vertices. Liskovec [40] 
found a recurrence for the number s, of strongly connected digraphs on n vertices, 
later simplified by Wright [72]. Let b, be defined recursively by 


n—-1 
Dn —_ gn(n-1) _ ry (") Qe) pe. b, =]. 
t=1 


Then 


This gives the sequence 
1, 1,18, 1606,565080,..., 


which is sequence A003030. 
Wright [73] also studied the number s;,, of strongly connected digraphs with n 
vertices and m edges. He showed that 


Santk = Py(n)n!, 


where P;, is polynomial in n of degree 3k — 1. Notice the similarity of the previous 
expression with equation (6.5) for connected graphs with excess k. The first values 
are 


P(n)= 3(n-2)(n-+3), 


1 
P)(n) = 5gg0 (2 — 2)(Sdat + 29TH" 271n? — 1937n — 1020). 


This was extended more recently in [48] to cover the range k = O(nlogn). Beyond 
this range a digraph is almost surely strongly connected. 


6.9 Unlabeled graphs 


An unlabeled graph is an isomorphism class of labeled graphs. The number of ways 
of labeling an unlabeled graph G is equal to n!/aut(G), where aut(G) is the number 
of automorphisms of G. Clearly counting unlabeled graphs is more demanding than 
counting their labeled counterparts, since symmetries must be taken into account. 
In this section we review briefly the theory of counting under symmetries, and also 
discuss the asymptotic enumeration of unlabeled graphs. 
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6.9.1 Counting graphs under symmetries 


The starting point of the theory is the orbit-counting lemma, also known as Burnside- 
Frobenius lemma. Let I be a permutation group acting on a finite set X = {1,...,n}. 
Then the number of orbits of the action of T on X is equal to 


1 


Tr y fix(o), 


ocl 


where fix() is the number of points fixed by o. 
The type of a permutation o is 1°12°2 ---n*", where s; is the number of i-cycles in 
the cycle decomposition of o. Its cycle index is a monomial in n variables defined as 


. — 751 S 
Z(O3X1, +++ yXn) =X} Xl! 


The cycle index polynomial of T is 


1 aE 
ZV SH jcc Xn) = Ir y Z(O3X1,---,Xn) = ra > Te. 


The cycle index polynomial can be used to count inequivalent colorings (in a very 
general sense) of combinatorial objects under symmetries. Let C be a set of “colors” 
and c = |C|. A coloring is a mapping f: X — C. Two mapping f,g are equivalent if 
there exists o € T such that g(x) = f(o(x)). That is, they are the same mapping up 
to a symmetry. The basic version of Pélya theorem is that the number of inequivalent 
colorings of X by colors in C under the action of I’ is equal to 


Z(G3c,...,c). 


Classical applications of the previous result are, for instance, the enumeration of 
necklaces (bicolored cycles) up to cyclic or dihedral symmetry, or counting the num- 
ber of ways of colorings the faces of a cube with a given number of colors, up to 
symmetries of the cube. 

A useful and important generalization is the following, also known as Redfield- 
Polya’s theorem. Let w: X — R be a weight function into a commutative ring R, 
which in the cases of interest will be either the ring of integers or a polynomial 
ring. A coloring is a mapping f: X — R. The weight of f is defined as w(f) = 
Tlxrex wf (x)). Notice that two colorings in the same orbit have the same weight. 
Then we have the following result. 


Theorem 6.9.1 (Refield-Pélya) Let @ be the set of orbits of X by R under the action 
of T. Then 


y w(f) =Z (r s: w(c), y we)... > wo") ‘ 


fEG cEc eeC cEC 


When all the weights are equal to 1, one recovers Pélya’s theorem. 
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We now apply Pélya’s theorem to counting unlabeled graphs. Let V = {1,...,n} 
and let EF be the set of all 2-subsets of V. A simple graph with vertex set V can be 
seen as a mapping f: E — {0,1}, such that f({x,y}) = 1 if and only if {x,y} is an 
edge. Two graphs are isomorphic if there exists a permutation of V mapping edges 
of one graph to edges of the other one. Let S,, be the symmetric group on V and let 
Si, = {o’: o €S,} be the group (isomorphic to S,,) acting on edges as 

o': {x,y} > {o(x), o(y)}- 


Then f,g: E — {0,1} correspond to isomorphic graphs if they are in the same orbit 
of the action of S’ on E. For instance, o € Ss has type (2'3!), then o’ has type 
(1'3!6!). If we perform all the calculations (not a pleasant task if done by hand) then 
we obtain 


Z(S53%1,%2,%3,%4,%5,---) = 
as (at? + 10x}x3 + 15xix3 + 200x133 + 20x1x3%6 + 30x2x3 + 24x32) , 
The number of unlabeled graphs with five vertices is then 
Z(S532,2,2,2,2) = 34. 
The counting sequence of unlabeled graphs, starting with n = 1, is 


1,2,4, 11,34, 156, 1044, 12346,... 


which is sequence A000088. We can also count edges; it suffices to takes as weights 
w(0) = 1,w(1) =x. Then the enumerator polynomial for edges is 


Z(Se1 $11 He ee) = 


L4+x+2x* +433 + 6x? + 6x° +6x° +4x! +28 +x? + x10. 


This is an example that shows the usefulness of the general weighted version. Many 
other examples can be found in [32]. The case of trees can be handled directly with 
the symbolic method (for rooted trees) and the dissymmetry theorem (for unrooted 
trees) that is discussed in Chapter 4 of this book. 


6.9.2 Asymptotics 


Let Y be a class of labeled graphs and Y% the corresponding class of unlabeled 
graphs. Suppose we have a result saying that almost all graphs in Y have no au- 
tomorphism as n — co. Then it follows that 
G, 
|| ~ | nl (6.10) 


n! 


Hence an estimate for |Y,,| provides an estimate for Y%,|. Of course one can ask for 
finer asymptotics and investigate the error term. 
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In order to prove such a result it is enough to show that 
E(aut(G)) > 1, n— ©, 
This is known in several important cases: 


e General graphs. As we mentioned in the introduction almost every labeled 
graph has no automorphism. This statement can be refined by taking into 
account also the number of edges. Let Gy and Un m be, respectively, the 
number of labeled and unlabeled graphs with n vertices and m edges. Notice 
that Grim = cap where N = ys and trivially Unm > Gan /n!. The following 
holds [10, Chap. 9]: 

If c > 1 is a constant and 
cnlogn<m< N—cnlogn, 


then 

Unm © Gam/n!. 
The condition m > cnlogn is necessary as otherwise almost surely there is 
more than one isolated vertex and there is non-trivial automorphism (the sec- 
ond inequality is also necessary by complementation). 


e Regular graphs. The fact that E(aut(G)) > 1 for d-regular graphs was first 
proved by Bollobas for fixed d, and then extended to d = o(n*) by McKay 
and Wormald; see the discussion in [70]. 


There are situations where the expected number of automorphisms is very large 
and the relation (6.10) does not hold. The first notable example is the case of trees. It 
is well-known that a random tree contains almost surely Qn subtrees isomorphic to 
any fixed subtree 7p. If we take 7p as the tree on three vertices rooted at the vertex of 
degree two, then 7p has a non-trivial automorphism that exchanges the two vertices 
of degree one. This action can be applied independently to any copy of 7p, and this 
implies that almost surely there are at least c” autormophisms, where c = 2°. 

This phenomenon is true more generally for addable minor-closed classes (Sec- 
tion 6.7), since again almost surely there are linearly many pendant copies of each 
connected graph in the class. For subcritical families there is a general result [17] 
saying the number of unlabeled graphs in the class is asymptotically 


ce 


u 


where y, is the unlabeled growth constant. Because almost surely the number of 
automorphisms is exponential, necessarily y, > y, the labeled growth constant. In 
some cases (for example for outerplanar graphs), y, has been determined. But for 
planar graphs this is a remarkable open problem. As we have seen in this case y © 
27.23, which is a lower bound for y,. The best upper bound for y, is 


Yu < 30.06, 


which is proved by encoding unlabeled planar graphs with an bits, and a@ + 4.91. The 
main obstacle for extending the techniques of [17] to planar graphs is the numeration 
of unlabeled planar graphs. 
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7.1 Introduction 


Many important sequences in combinatorics are known to be log-concave or uni- 
modal, but many are only conjectured to be so although several techniques using 
methods from combinatorics, algebra, geometry and analysis are now available. Stan- 
ley [90] and Brenti [25] have written extensive surveys of various techniques that can 
be used to prove real-rootedness, log-concavity or unimodality. After a brief intro- 
duction and a short section on probabilistic consequences of real-rootedness, we will 
complement [25, 90] with a survey over new techniques that have been developed, 
and problems and conjectures that have been solved. I stress that this is not a compre- 
hensive account of all work that has been done in the area since op. cit.. The selection 
is certainly colored by my taste and knowledge. 
If AS = {ax}h_p is a finite sequence of real numbers, then 


e is unimodal if there is an index 0 < j <n such that 


a9 Se SGj-] Sj = jz) 2+ [> Ay. 


e of is log-concave if 


a; >aj-1aj41, foralll <j<n. 


e the generating polynomial, p.y(x) := a9 + ajx+---+ a,x", is called real- 
rooted if all its zeros are real. By convention we also consider constant poly- 
nomials to be real-rooted. 


We say that the polynomial p(x) = Lit) ax" has a certain property if o = {a,}/_9 
does. The most fundamental sequence satisfying all of the properties above is the nth 
row of Pascal’s triangle {(7) }i_p- Log-concavity follows easily from the explicit 
formula (7) =n!/k!(n —k)!: 


(e+ 1)@—k+1) 


(1) (via) k(n—k) 


The following lemma relates the three properties above. 


Lemma 7.1.1 Let / = {ax}{_, be a finite sequence of nonnegative numbers. 
e If p(x) is real-rooted, then the sequence @' := {ax/ (it) }f_9 is log-concave. 
e If x’ is log-concave, then so is &. 


e If & is log-concave and positive, then & is unimodal. 
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Proof. Suppose p,(x) is real-rooted. Let ax = (7) be, for 1 < k <n. By the Gauss— 
Lucas theorem below, the polynomial 


nk n—1 —] 
Pot) =) (jou =) (" ‘ ) rua! (7.1) 


k=0 


is real-rooted. The operation 


“(n 

spa iis) = (1) boas 72) 
imo \k 

preserves real-rootedness. Let 1 < j <n— 1. Applying the operations (7.1) and (7.2) 

appropriately to p,7(x), we end up with the real-rooted polynomial 


bj-1 + 2b jx +b j412", 


and thus bi > b;_,bj+1. This proves the first statement. 

The term-wise (Hadamard) product of a positive and log-concave sequence and 
a log-concave sequence is again log-concave. Since {(7) }i_9 is positive and log- 
concave, the second statement follows. 

The third statement follows directly from the definitions. | 


Example 7.1.2 Natural examples of log-concave polynomials that are not real- 
rooted are the q-factorial polynomials, 


[a]q! = [nq a— 1]q--- 2]q- (a. 
where |k|q = 1+q+-:-+q""'. The polynomial |n],! is the generating polynomial 
for the number of inversions over the symmetric group Gy: 


t= yan, 


TEGn 


where 
inv() =|{1 <i<j<n:a(i) >x(j)}H, 


see [94]. The easiest way to see that |n|q! is log-concave is to observe that {k\q is 
log-concave. Log-concavity of |n],! then follows from the fact that if A(x) and B(x) 
are generating polynomials of positive log-concave sequences, then so is A(x)B(x), 


see [90]. 


Example 7.1.3 Examples of unimodal sequences that are not log-concave are the 
q-binomial coefficients 


fl > meee 


These are polynomials with nonnegative coefficients 


i = a(n, k) +a1(n, kg to + ayn (gee, (7.3) 
q 
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which are unimodal and symmetric. There are several proofs of this fact, see [90]. 
For example the Cayley—Sylvester theorem, first stated by Cayley in the 1850’s and 


proved by Sylvester in 1878, implies unimodality of (7.3), see [90]. However al a= 


1+q4+2q?+¢ +44, which is not log-concave. 
For a proof of the following fundamental theorem we refer to [82]. 


Theorem 7.1.4 (The Gauss-Lucas theorem) Let f(x) € C[x] be a polynomial of 
degree at least one. All zeros of f’(x) lie in the convex hull of the zeros of f(x). 


Example 7.1.5 Let {S(n,k)}_,) be the Stirling numbers of the second kind, 
see [94]. Then S(n,k) := k!S(n,k) counts the number of surjections from [n] := 
{1,2,...,n} to [k]. For a surjection f : [n+ 1] > |[k], let j = f(n +1). Condition- 
ing on whether |f—!({j})| = 1 or |f~!({j})| > 1, one sees that 


S(n+1,k) =kS(n,k—-1)+kS(n,k), foralll <k<n+l. (7.4) 


Let En(x) = ¥"_, S(n,k)x*. Then (7.4) translates as 


Enst(x) = xEn(x) +x(4+ DEL (x) = ce ((+ L)En(x)). 


By induction and the Gauss—Lucas theorem, we see that E,,(x) is real-rooted, and 
that all its zeros lie in the interval |—1,0] for all n > 1. Later, in Example 7.7.2 we 
will see that the operation of dividing the kth coefficient by k!, for each k, preserves 
real-rootedness. Hence also the polynomials Y7_ S(n,k)x*, n > 1, are real-rooted. 


A generalization of finite nonnegative sequences with real-rooted generating 
polynomials is that of Polya frequency sequences. A sequence {a;}7 9 C R is a 
Polya frequency sequence (PF for short) if all minors of the infinite Toeplitz ma- 
trix (ai-j)7-j=0 are nonnegative. In particular, PF sequences are log-concave. PF se- 
quences are characterized by the following theorem of Edrei [42], first conjectured 
by Schoenberg. 


Theorem 7.1.6 A sequence {ax}~_ © R of real numbers is PF if and only its gen- 
erating function may be expressed as 


y “eS cre" TT (1 a aux) TY — Bex), 
k=O. 


k=0 k=0 
where C,a>0,mEN, a, By > O for allk € N, and Yeo (Qk + Be) < =. 


Hence a finite nonnegative sequence is PF if and only its generating polynomial is 
real-rooted. This was first proved by Aissen, Schoenberg and Whitney [1]. Theorem 
7.1.6 provides, at least in theory, a method of proving combinatorially that a com- 
binatorial polynomial with nonnegative coefficients is real-rooted. Namely to find a 
combinatorial interpretation of the minors of (a;_ Dij=0- This method was used by 
e.g. Gasharov [52] to prove that the independence polynomial of a (3 + 1)-free graph 
is real-rooted. For more on PF sequences in combinatorics, see [24]. 
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7.2 Probabilistic consequences of real-rootedness 


Below we will explain two useful probabilistic consequences of real-rootedness. For 
further consequences, see Pitman’s survey [79]. If X is a random variable taking 
values in {0,...,n}, let a, = P[X =k] for0 <k <n, and let 


Px(t) =ao +ait +--+ +ant", 


be the partition function of X. Then X has mean 


yw =E(x] = YP =A = ph (1), 
k=0 


and variance 


Var(X)) = E[X*] — uw? = px (1) + py (1) — px (1). 


The following theorem of Bender [4] has been used on numerous occasions to prove 
asymptotic normality of combinatorial sequences, see e.g. [4, 5, 8]. 


Theorem 7.2.1 Let {X,}°_, be a sequence of random variables taking values in 
{0,1,...,n} such that 


1. px, (t) is real-rooted for all n, and 


2. Var(Xn) — ©. 
Then the distribution of the random variable 
X, — EX 
Var (Xn) 


converges to the standard normal distribution N(0,1) as n — ©. 


Example 7.2.2 Let X, be the random variable on the symmetric group Gy counting 
the number of cycles in a uniform random permutation. Since the number of permu- 
tations in Gy, with exactly k cycles is the signless Stirling number of the first kind 
c(n,k) (see [94]), 
1 
Prilt) = x(e+ 1) (etn), 
Thus X, has mean Hy, = 1+1/2+-+-+1/n and variance 
2 ‘ Z 
0, =H,-)ik°. 
k=1 
Hence the distribution of the random variable 
Xn — Ay 
On 


converges to the standard normal distribution N(0,1) as n + ©. 
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For more examples using Theorem 7.2.1, see [4], and for recent examples, see 
[5, 8]. 

A simple consequence of Lemma 7.1.1 is that if a polynomial ap + ajx +--+ 
a,x" has only real and nonpositive zeros, then there is either a unique index m such 
that ad = max, a,, or two consecutive indices m+ 1/2 (whence m is a half-integer) 
such that d41/2 = max,ax. The number m = m({ax}i_9) is called the mode of 
{ax }{_- A theorem of Darroch [40] enables us to easily compute the mode. 


Theorem 7.2.3 Suppose {a;}/_ is a sequence of nonnegative numbers such that the 
polynomial p(x) = ag +.a,x+-+++a)x" is real-rooted. If m is the mode of {ax}{_. 
and | := p'(1)/p(1) its mean, then 

[H] <m< [ul]. 


Applying Theorem 7.2.3 to the signless Stirling numbers of the first kind {c(n,k)}7_, 
(Example 7.2.2), we see that 


[Hn] < m({e(n,k) rai) S [An]. 


7.3 Unimodality and y-nonnegativity 


We say that the sequence {h,}4_, is symmetric with center of symmetry d/2 if 
hy = ha_ for all 0 < k < d. A property called y-nonnegativity, which implies sym- 
metry and unimodality, has recently been considered in topological, algebraic and 
enumerative combinatorics. 

The linear space of polynomials h(x) = Y4_9 hyx* € R[x], which are symmetric 
with center of symmetry d/2, has a basis 


Be = {x Rage kyla) 


Tf h(x) = Dee! ext 1 +x), we call {%}44>! the y-vector of h. Since the bino- 
mial numbers are unimodal, havine a nonnegative y-vector implies unimodality of 
{hy }i_o. If the y-vector of h is nonnegative, then we say that / is y-nonnegative. 
Let re be the convex cone of polynomials that have nonnegative coefficients when 
expanded in By. Clearly 


"rt c= {fg: ferrand gem} or. (7.5) 
Remark 7.3.1 Suppose /(x) = Y4_)/yx* € R[x] is the generating polynomial of a 


nonnegative and symmetric sequence with center of symmetry d/2. If all its zeros 
are real, then we may pair the negative zeros into reciprocal pairs 


h(x) =A“ [e+ 6) (e+1/6,) =a T]( + 1/0;—2)x), 


i=1 
where A > 0. Since x and (1 +x)? + (6;+1/0;—2)x are polynomials in I", we see 
that 4 is y-nonnegative by (7.5). 
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7.3.1 An action on permutations 


There is a natural Z5-action on Gp, first considered in a modified version by Foata 
and Strehl [49], which has been used to prove y-nonnegativity. Let 7 = aja2---ay € 
G,, be a permutation written as a word (2(i) = a,), and set a9 = dn41 =n +1. If 
k € |n], then a, is a 


e valley if az_1 > ay < ag41, 

e peak if ap_1 < ay > aga, 

e double ascent if aj_, < ag < ay41, and 

e double descent if ap_; > ay > ag44. 
Define functions @, : G, > G,, x € [n], as follows: 


e If x is a double descent, then @,(z) is obtained by moving x into the slot 
between the first pair of letters a;,a;, 1 to the right of x such that aj <x < aj443 


e Ifxisadouble ascent, then @,(7) is obtained by moving x to the slot between 
the first pair of letters a;,aj+1 to the left of x such that aj > x > aj44; 


e If xis a valley ora peak, then 9,(z) = 7. 


There is a geometric interpretation of the functions @,, x € |n], first considered 
in [87]. Let 7 = a,az---ay, € Gy, and imagine marbles at the points (i,a;) € N x 
N, for i =0,1,...,2+1. For i=0,1,...,1 connect (i,a;) and (i+ 1,aj;41) with a 
wire. Suppose gravity acts on the marbles, and that x is not at an equilibrium. If 
x is released it will slide and stop when it has reached the same height again. The 
resulting permutation is @,(7), see Figure 7.1. 

The functions @, are commuting involutions. Hence for any subset S C |n], we 
may define the function @s : G, > G, by 


@s(m) =] @:(z). 


xeS 


Hence the group Z3 acts on G, via the functions @s, S C [n]. For example 
,2,3,7,8} (573148926) = 857134926. 


For 2 € G,, let Orb(7) = {g(z) : g € Z5} be the orbit of 7 under the action. 
There is a unique element in Orb(z) that has no double descents, which we denote 
by %. 


Theorem 7.3.2 Let 1 = ajaz+--dn € Gy. Then 


des(o) — ea) (7 apg t ot ede h) _ Pele) (1 eat eee) 
o€Orb(z) 


where des(m) = |{i € [n] : aj > aj41}| and peak(z) = |{i € [n] : aj < aj > ai41}|. 
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Figure 7.1 
Graphical representation of 7 = 573148926. The dotted lines indicate where the dou- 
ble ascents/descents move to. 


Proof. If x is a double ascent in a then des(@,()) = des(z) + 1. It follows that 


Y xis) = x81 4x94, 
o€Orb(z) 


where a is the number of double ascents in 7. If we delete all double ascents from 7 
we get an alternating permutation 


n+1>b, <bo >b3< ++) > Dna < ntl, 
with the same number of descents. Hence n — a = 2des(#) + 1. Clearly des(7) 


peak(z) and the theorem follows. | 


For a subset T of G, let 


= y des) 


neT 


Corollary 7.3.3 [fT © Gy is invariant under the Z-action, then 


[n/2| 
ae ¥(T ba 0 ee aes coe, 


where 
¥(T) =2-"t14 {9 © T : peak(x) = i} |. 
In particular A(T,x) is Y-nonnegative. 
Proof. tis enough to prove the theorem for an orbit of a permutation z € G,. Since 


the number of peaks is constant on Orb(z) the equality follows from Theorem 7.3.2. 
a 
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Example 7.3.4 Recall that the Eulerian polynomials are defined by 


= y xdes(a) +1 (7.6) 
NEG 
see [94]. By Corollary 7.3.3, 
[n/2| ; : 
An(x) /x = y Ynix' (1 Sa ae 
i=0 


where 


Yi = 2-117! {7 © Gy : peak(x) = i} |. 


Example 7.3.5 This example is taken from [18]. The stack-sorting operator S may 
be defined recursively on permutations of finite subsets of {1,2,...} as follows. If w 
is empty, then S(w) := w. If w is nonempty, write w as the concatenation w = LmR 
where m is the greatest element of w, and L and R are the subwords to the left and 
right of m, respectively. Then S(w) := S(L)S(R)m. 

If o,t € Gy are in the same orbit under the Z-action, then it is not hard to prove 
that S(o) = S(t), see [18]. Let rE N. A permutation m € G,, is said to be r-stack 
sortable if, S’(2) = 12---n. Denote by Gy, the set of r-stack sortable permutations 
in Gy. Hence Gj, is invariant under the Z5-action for alln,r € N, so Corollary 7.3.3 
applies to prove that for alln,r EN 


[n/2! 
A(G!;x) a 7; ( (G")x Ease ito Hote 


where 


1(Gi,) =2-" i {ar © Gr: peak(m) = i}. 


Unimodality and symmetry of A(G!,;x) was first proved by Bona [7]. Bona conjec- 
tured that A(G);x) is real-rooted for all n,r € N. This conjecture remains open for 
all3 <r<n-—3, see [18]. 

More generally, if A C Gy, then the polynomial 


y des) 


TES 
S(m)EA 


is Y-nonnegative. 


Postnikov, Reiner and Williams [81] modified the Z5-action to prove Gal’s con- 
jecture (see Conjecture 7.3.8) for so-called chordal nestohedra. 

In [88], Shareshian and Wachs proved refinements of the y-positivity of Eulerian 
polynomials. Let 


n 


Aap) Aulapps = Yq pee eon), 
k=0 o€Gn 
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where 
exc(o) = |{i: o(7) > i}I, 
fix(o) = |{i: o(i) =i}|, and 
maj(o)= Yk. 


i:0(i)>o(i+1) 


Theorem 7.3.6 Let By = {t*(1+1)* 2ky Lard 


1. The polynomial Ano(q,p,q"'t) has coefficients in N(q, p] when expanded in 
Bn. 


2. If 1 <k <n, then Ayx(q,1,q_'t) has coefficients in N[g| when expanded in 
Bn—k- 


3. The polynomial A,(q,1,1,q~'t) has coefficients in N[q| when expanded in 
Bn-1. 


Gessel [53] has conjectured a fascinating property that resembles y-nonnegativity 
for the joint distribution of descents and inverse descents: 


Conjecture 7.3.7 (Gessel [53, 18, 78]) [fn is a positive integer, then there are non- 
negative numbers Cy(k, j) for all k, j € N such that 


Y Ay = YP eae NeryoVtay et, 7.7) 


TEGn k,jeN 
k4+2j<n-1 


The existence of integers c,(k, j) satisfying (7.7) follows from symmetry properties, 
see [78]. The open problem is nonnegativity. 


7.3.2 y-nonnegativity of h-polynomials 


In topological combinatorics the y-vectors were introduced in the context of 
face numbers of simplicial complexes [15, 51]. The f-polynomial of a (d — 1)- 
dimensional simplicial complex A is 


d 
=¥ fide 
t=O 


where f(A) is the number of k-dimensional faces in A, and f_;(A) := 1. The h- 
polynomial is defined by 


= ¥ mca) xk = (1—x)“f,(x/(1—x)), or equivalently, (7.8) 


fa(x) = (1 “+ 2)hale/ “)) 
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Hence f,(x) and ha(x) contain the same information. If A is a (d — 1)-dimensional 
homology sphere, then the Dehn—Sommerville relations (see [91]) tell us that A, (x) 
is symmetric, so we may expand it in the basis By. Recall that a simplicial complex A 
is flag if all minimal non-faces of A have cardinality two. Motivated by the Charney— 
Davis conjecture below, Gal made the following intriguing conjecture: 


Conjecture 7.3.8 (Gal [51]) If A is a flag homology sphere, then ha(x) is y- 
nonnegative. 


Gal’s conjecture is true for dimensions less than five, see [51]. If ha(x) is sym- 
metric with center of symmetry d/2, then ha(—1) = 0 if d is odd, and ha(—1) = 
(-1)4/?y, /2(A) if d is even. Hence Gal’s conjecture implies the Charney—Davis con- 
jecture: 


Conjecture 7.3.9 (Charney—Davis [30]) /f A is a flag (d — 1)-dimensional homol- 
ogy sphere, where d is even, then (—1)4/2h,(—1) is nonnegative. 


Postnikov, Reiner and Williams [81] proposed a natural extension of Conjec- 
ture 7.3.8. 


Conjecture 7.3.10 [fA and A’ are flag homology spheres such that A’ geometrically 
subdivides A, then the y-vector of A’ is entry-wise larger or equal to the y-vector of 
A. 


Conjecture 7.3.10 was proved for dimensions < 4 in a slightly stronger form by 
Athanasiadis [3]. In [3], Athanasiadis also proposes an analog of Gal’s conjecture 
for local h-polynomials. 


7.3.3 Barycentric subdivisions 


The collection of faces of a regular cell complex A are naturally partially ordered 
by inclusion; If F and G are open cells in A, then F < G if F is contained in the 
closure of G, where we assume that the empty face is contained in every other face. 
A Boolean cell complex is a regular cell complex such that each interval [0, F] = 
{Gé€A:G<F} is isomorphic to a Boolean lattice. Hence simplicial complexes are 
Boolean. The barycentric subdivision, sd(A), of a Boolean cell complex, A, is the 
simplicial complex whose (k — 1)-dimensional faces are strictly increasing flags 


Fi<hj<::-< kf, 


where F; is a nonempty face of A for each 1 < j <k. The f-polynomials and h- 
polynomials for cell complexes are defined just as for simplicial complexes. 

Brenti and Welker [27] investigated positivity properties, such as real-rootedness 
and y-positivity, of the h-polynomials of complexes under the operation of taking 
barycentric subdivisions. This was done by using analytic properties (obtained in 
[16, 27]) of the linear operator that takes the f-polynomial of a Boolean complex 
to the f-polynomial of its barycentric subdivision. These analytic properties will be 
discussed in Section 7.7.1. In this section we describe the topological consequences 
of the analytic properties. 
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Let & : R[x] + R[x] be the linear operator defined by its image on the binomial 
basis: 


2 ee a). Be 1) ek 1) 
6({)=* forall kN, where (#) = a . 


The operator & appears in several combinatorial settings. Using the binomial theo- 
rem one sees 


=¥ Fo) ee (7 ty)rtl’ 


It follows e.g. from the theory of P-partitions (or from (7.4) and induction) that 


E(x") = E,(x) = y k!S(n,k)x*, for alln > 1, 
k=l 


where {S(n,k)}/_9 are the Stirling numbers of the second kind, see [94, 102]. 
The following lemma was proved by Brenti and Welker [27]. 


Lemma 7.3.11 For any Boolean cell complex A, 
foa(a) = © (fa): 


Proof. — By definition 


feacay (x) = Y We (x) 
FEA 
dimF+1 
We(x)= Yo ALOK <- <R=FY}I, 
k=1 


if F £0. Since A is Boolean, there is a one-to-one correspondence between flags 
O<Fi<-:-<Rm=F,1<k<dimF +1, and ordered set-partitions of [n], where 
n=dimF +1. Hence Wr(x) = E,(x) = &(x”), and the lemma follows. | 


Lemma 7.3.12 Let A be a (d —1)-dimensional Boolean cell complex. If ha(x) is 
symmetric, then so is hgq(a)(X)- 


Proof. By (7.8), ha(x) is symmetric if and only if (—1)4f4(—1 —x) = fa(x). Let: 
Rx] > R[x] be the algebra automorphism defined by J(x) = —1—-. It was observed 
in [16, Lemma 4.3] that 

lo€ =€ol, (7.9) 


from which the lemma follows. | 
Corollary 7.3.13 [27] Let A be a Boolean cell complex. If the h-polynomial of A has 


nonnegative coefficients, then all zeros of haa) (x) are nonpositive and simple. 
If hy(x) is also symmetric, then hgaia) (x) is Y-nonnegative. 
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Proof. — The first conclusion is immediate from Theorem 7.7.11, Lemma 7.3.11 and 
(7.8). The second conclusion follows from Remark 7.3.1 and Lemma 7.3.12. | 


The second conclusion of Corollary 7.3.13 was strengthened in [71], where it 
was shown that with the same hypothesis, the y-vector of sd(A) is the f-vector of a 
balanced simplicial complex. 

If A is a Boolean cell complex and k is a positive integer, let sd‘ (A) be the sim- 
plicial complex obtained by a k-fold application of the subdivision operator sd. Most 
of the following corollary appears in [27]. 


Corollary 7.3.14 Let A be a (d—1)-dimensional Boolean cell complex with reduced 
Euler characteristic ¥(A), where d > 2. There exists a number N(A) such that 


1. all zeros of hggn(a) (x) are real and simple for alln > N(A), 


2. if (—1)4-!%(A) = 0, then all zeros of hsqr(a)(x) are nonpositive and simple for 
alln > N(A), 


3. if (—1)4-!%(A) <0, then all zeros of hsan (a) (x) except one are nonpositive and 
simple for alln > N(A). 


Moreover 


tim im sale A)(x ) = fa—1(A)pa(x), (7.10) 


where pq(x) is the unique monic degree d eigenpolynomial of & (see Theorem 
7.7.12). 


Proof. — The identity (7.10) follows from the proof of Theorem 7.7.12 by choosing 
f = fa(x)/fa-1(A). By Theorem 7.7.12, all zeros of pg(x) are real, simple and lie in 
the interval [—1,0]. In view of (7.10) all zeros of f,qn(q)(x) will be real and simple 
for n sufficiently large. The same holds for /gqn(q) (x) by (7.8). 

Assume (—1)4—'%(A) > 0. By Theorem 7.7.12, pa(0) = pa(—1) = 0. Since 
foan(a)(-1) = fa(-1) = —X% (A), we see by (7.10) that for all n sufficiently large all 
zeros of f,an(a) (x) are simple and lie in [—1,0) (since f<qr() (x) has the correct sign 
to the left of —1). By (7.8) this is equivalent to (2). Statement (3) follows similary. 


Corollary 7.3.15 Let A be a (d—1)-dimensional Boolean cell complex such that 
h(x) is symmetric and (—1)4~!%(A) > 0. Then there is a number N(A) such that 
hgan(a) (x) is Y-nonnegative whenever n > N(A). 


Proof. Combine Remark 7.3.1, Lemma 7.3.12 and Corollary 7.3.14. a 


7.3.4 Unimodality of 1*-polynomials 


Let P C R” be an m-dimensional integral polytope, i.e., all vertices have integer 
coordinates. Ehrhart [43, 44] proved that the function 


i(P.r) = |rPN 2", 
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which counts the number of integer points in the r-fold dilate of P, is a polynomial 
in r of degree m. It follows that we may write 


= hi(P) +h} (P)x +--+ he (P)x" 
Y i(P,r)x" = ol?) + ae n(P) (7.11) 


r=0 


Stanley [89] proved that the coefficients of the polynomial, (x), in the numerator of 
(7.11) are nonnegative, and Hibi [59] conjectured that (x) is unimodal whenever it 
is symmetric. Hibi [59] proved the conjecture for n < 5. However Payne and Mustata 
[69, 74] found counterexamples to Hibi’s conjecture for each n > 6. Let us mention 
a weaker conjecture that is still open. An integral polytope P is Gorenstein if h}(x) 
is symmetric, and P is integrally closed if each integer point in rP may be written as 
a sum of r integer points in P, for all r > 1. 


Conjecture 7.3.16 (Ohsugi-Hibi [73]) Jf P is a Gorenstein and integrally closed 
integral polytope, then h(x) is unimodal. 


Inspired by work of Reiner and Welker [84], Athanasiadis [2] provided conditions 
on an integral polytope P that imply that 43(x) is the -polynomial of the bound- 
ary complex of a simplicial polytope. Hence, by the g-theorem (see [91]), (x) is 
unimodal. Athanasiadis used this result to prove the following conjecture of Stanley. 
An integer stochastic matrix is a square matrix with nonnegative integer entries 
having all row and column sums equal to each other. Let H,,(r) be the number of 
n Xn integer stochastic matrices with row and column sums equal to r. The func- 
tion r++ H,,(r) is the Ehrhart polynomial of the integral polytope P,, of real doubly 
stochastic matrices. Stanley [91] conjectured that hp, (x) is unimodal for all positive 
integers n, and Athanasiadis’ proof of Stanley’s conjecture was the main application 
of the techniques developed in [2]. Subsequently Bruns and R6mer [28] generalized 
Athanasiadis results to the following general theorem. 


Theorem 7.3.17 Let P be a Gorenstein integral polytope such that P has a regular 
unimodular triangulation. Then hp(x) is the h-polynomial of the boundary complex 
of a simplicial polytope. In particular, h}(x) is unimodal. 


7.4 Log-concavity and matroids 


Several important sequences associated to matroids have been conjectured to be log- 
concave. Progress on these conjectures have been very limited until the recent break- 
through of Huh and Huh-Katz [61, 62]. Recall that the characteristic polynomial 
of a matroid M is defined as 
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where Ly is the lattice of flats, u its M6bius function, r is the rank function of M 
and {(—1)‘w,(M vee are the Whitney numbers of the first kind. The sequence 
{w;(M)};_9 is nonnegative, and it was conjectured by Rota and Heron to be uni- 


modal. Welsh later conjectured that (,(M)} is log-concave. It is known that 
a4m(1) = 0. Define the reduced characteristic polynomial by 


r-1 


Xu (x) = Xu(x)/(x—1) =: y (=1), je. 
k=0 


Note that if {v, (My ye is log-concave, then so is {we(M) see [90]. 


Theorem 7.4.1 (Huh—-Katz [62]) /f M is representable over some field, then the se- 
quence {v,(M) ee is log-concave. 


Since the chromatic polynomial of a graph is the characteristic polynomial of a 
representable matroid we have the following corollary: 


Corollary 7.4.2 (Huh [61]) Chromatic polynomials of graphs are log-concave. 


Let 

r(M) 

f(x) = Y (-1)E fe (Mx, 

k=0 
where f;(M) is the number of independent sets of M of cardinality k. Hence f(x) 
is the (signed) f-polynomial of the independence complex of M. Now, fiy(x) = 
Xuxe(x), where M x e is the free coextension of M, see [29, 65]. Also if M is 
representable over some field, then so is M x e. Hence the following corollary is 
a consequence of Theorem 7.4.1. 


Corollary 7.4.3 If M is representable over some field, then { fe(M) ye” is log- 
concave. 


This corollary, first noted by Lenz [65], verifies the weakest version of Mason’s con- 
jecture below for the class of representable matroids. 


Conjecture 7.4.4 (Mason) Let M be a matroid and n = f\(M). The following se- 
quences are log-concave: 


r(M) 
r(M M n 
(MOND, RCO, ana Lacony(P)) 
k=0 
The proofs in [61, 62] use involved algebraic machinery that falls beyond the 
scope of this survey. It is unclear if the method can be extended to the case of non- 
representable matroids. 
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7.5 Infinite log-concavity 


Consider the operator & on sequences ./ = {ax}. C R defined by 7(./) = 
{bx}, Where 


bo =ay and by =ax—ag_jay41, fork>1. 


This definition makes sense for finite sequences by regarding these as infinite se- 
quences with finitely many nonzero entries. Hence a sequence . is log-concave if 
and only if “(./) is a nonnegative sequence. A sequence is k-fold log-concave if 
L/(A) is a nonnegative sequence for all 0 < j <k. A sequence is infinitely log- 
concave if it is k-fold log-concave for all k > 1. Although similar notions were stud- 
ied by Craven and Csordas [37, 38], the following questions asked by Boros and Moll 
[13] spurred the interest in infinite log-concavity in the combinatorics community: 


(A) FormeN, let {de(m)}/.9 be defined by 


awe B(2) 090) 


Is the sequence {d¢(m)}/"_9 infinitely log-concave? 
(B) Forn €N, is the sequence { (7) }(_p infinitely log-concave? 


Question (A) is still open. However Chen et al. [31] proved 3-fold log-concavity of 
{d;(m)}/_9 by proving a related conjecture of the author that implies 3-fold log- 
concavity of {dp(m)}7_9, for each m € N, by the work of Craven and Csordas [38]. 
In connection to (B), Fisk [47], McNamara—Sagan [68], and Stanley [95] in- 
dependently conjectured the next theorem from which (B) easily follows. We may 
consider to be an operator on the generating function of the sequence, i.e., 


ee e at) = VF (ag = ay—sagy1)3*. 
k=0 


k=0 


Theorem 7.5.1 [19] If f(x) = LiLo a,x* is a polynomial with real and nonpositive 
zeros only, then so is £(f). In particular, the sequence {ax};_¢ is infinitely log- 
concave. 


The proof of Theorem 7.5.1 uses multivariate techniques, and will be given in 
Section 7.9.4. 

There is a simple criterion on a nonnegative sequence ./ = {ax}(°_, that guaran- 
tees infinite log-concavity [38, 68]. Namely 


az > ray_\dx41, forallk >1, 
where r > (34+ /5)/2. 
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McNamara and Sagan [68] conjectured that the operator preserves the class 
of PF sequences. In particular they conjectured that the columns of Pascal’s triangle 
{ (Py 7-o» Where k €N, are infinitely log-concave. In [20], Chasse and the author 
found counterexamples to the first mentioned conjecture and proved the second. They 
considered PF sequences that are interpolated by polynomials, i.e., PF sequences 
{p(k)}@_9 Where p is a polynomial, and asked when classes of such sequences are 
preserved by 2. 

Let Y be the following class of PF sequences that are interpolated by polynomi- 
als: 


{{p(k) }€9 € PF: p(x) € R[x] and p(—j) = p(—j +1) =0 for some j € {0,1,2}}. 


Theorem 7.5.2 [20] The operator & preserves the class Y. In particular each se- 
quence in is infinitely log-concave. 


Note that for each k € N, Ly *_o € #. The following corollary solves the above 
mentioned conjecture of McNamara and Sagan. 
Corollary 7.5.3. The columns of Pascal’s triangle are infinitely log-concave, i.e., for 


each k EN, the sequence £0) 7-0 bs infinitely log-concave. 


Let us end this section with an interesting open problem posed by Fisk [47]. 


Problem 7.5.4 Suppose all zeros of V7 a,x" 


of 


are nonpositive. If d € N, are all zeros 


n 
>? det(ai+i-j)4 ,=0 : a 
k=0 
where a; = 0 ifi ¢ {0,...,n}, nonpositive? 


Hence the case d = | of Problem 7.5.4 is Theorem 7.5.1. 


7.6 The Neggers—Stanley conjecture 


It is natural to ask if the real-rootedness of the Eulerian polynomials may be extended 
to generating polynomials of linear extensions of any poset. Define a labeled poset 
to be a poset of the form P = ([n],<p), where n is a positive integer. The Jordan-— 
Holder set of P, 


L(P) = {o € G,:i< j whenever o(i) <p o(J)}, 


is the set of all linear extensions of P. Here < denotes the usual order on the integers. 
The P-Eulerian polynomial is defined by 


Wp(x) = y des(o) +1 
o€L(P) 
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Figure 7.2 
Counterexamples to Neggers conjecture (left) and the Neggers—Stanley conjecture 
(right), taken from [97]. 


Recall that P is naturally labeled if i < j whenever i <p j. Neggers [70] conjec- 
tured in 1978 that Wp(x) is real-rooted for any naturally labeled poset P, and Stanley 
extended the conjecture to all labeled posets in 1986, see [24, 25, 102]. Counterex- 
amples to Stanley’s conjecture were first found by the author in [14], and shortly 
thereafter naturally labeled counterexamples were found by Stembridge in [97], see 
Figure 7.2. 

However, this does not seem to be the end of the story. Recall that a poset P is 
graded if all maximal chains in P have the same size. 


Theorem 7.6.1 (Reiner and Welker [84]) /f P is a graded and naturally labeled 
poset, then Wp(x) is unimodal. 


Reiner and Welker proved Theorem 7.6.1 by associating to P a simplicial polytope 
whose h-polynomial is equal to Wp(x)/x, and then invoking the g-theorem for sim- 
plicial polytopes. 

Theorem 7.6.1 was refined in [15, 18] to establish y-nonnegativity for the P- 
Eulerian polynomials of a class of labeled posets that contain the graded and naturally 
labeled posets. Let E(P) = {(i, j) : j covers i} be the Hasse diagram of a labeled 
poset P. Define a function € : E(P) > {—1, 1}, by 


— 1 if i< j, and 
e(i,j) = as: 
-1 ifj<i. 
A labeled poset P is sign-graded if for all maximal chains x9 <p x) <p--: <p x, in 


P, the quantity 


k 
r= Yi e(i-1,m) 
j=1 


L 
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is the same, see Figure 7.3. Note that a naturally labeled poset is sign-graded if and 
only if it is graded. 


—— m— f — 


N—A—w—A 


Figure 7.3 
A sign-graded poset of rank 1. 


Theorem 7.6.2 If P is sign-graded, then Wp(x) is y-nonnegative. 


Two proofs are known for Theorem 7.6.2. The first proof [15] uses a partitioning 
of £(P) into Jordan—Hdlder sets of refinements of P for which y-positivity is easy to 
prove. The second proof [18] uses an extension to £(P) of the Z4-action described 
in Section 7.3.1. 

Here are two questions left open regarding the Neggers—Stanley conjecture. 


Question 7.6.3 Are the coefficients of P-Eulerian polynomials log-concave or uni- 
modal? 


Question 7.6.4 Are P-Eulerian polynomials of graded (or sign-graded) posets real- 
rooted? 


The work in [15] was generalized by Stembridge [98] to certain Coxeter cones. 
Let ® be a finite root system in a real Euclidian space V with inner product (-,-). A 
Coxeter cone is a closed convex cone of the form 


A(W) ={weV: (u,B) > 0 for all B cP}, 


where ¥ C ®. This cone is a closed union of cells of the Coxeter complex defined by 
®, so it forms a simplicial complex that we identify with A(‘Y). A labeled Coxeter 
cone is a cone of the form 


A(P,A) = {u € AWW) : (u,B) > 0 for all B EY with (A,B) <0}, 


where A(V) is a Coxeter cone and A € V. Hence A(W,/) may be identified with a 
relative complex inside A(Y). When © is crystallographic, Stembridge defines what 
it means for a (labeled) Coxeter cone to be graded. In type A, graded labeled Coxeter 
cones correspond to sign-graded posets. 


Theorem 7.6.5 (Stembridge [98]) The h-vectors of graded labeled Coxeter cones 
are Y-nonnegative. 
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7.7 Preserving real-rootedness 


If a sequence of polynomials satisfies a linear recursion, then to prove that the poly- 
nomials are real-rooted it is sufficient to prove that the defining recursion “preserves” 
real-rootedness. Hence it is natural, from a combinatorial point of view, to ask which 
linear operators on polynomials preserve real-rootedness. This question has a rich 
history that goes back to the work of Jensen, Laguerre and Pélya, see the survey 
[39]. In his thesis, Brenti [24] studied this question focusing on operators occurring 
naturally in combinatorics. 

Let us recall Pélya and Schur’s [80] celebrated characterization of diagonal oper- 
ators preserving real-tootedness. A sequence A = {Ax }7°_, of real numbers is called 
a multiplier sequence (of the first kind), if the linear operator T, : R[x] > R[x] 
defined by 

Ty (x*) =Ayx*, kEN, 


preserves real-rootedness. 


Theorem 7.7.1 (Pélya and Schur [80]) Let A = {A }f) be a sequence of real 
numbers, and let 


= A, 
Gale) = Vo, 


be its exponential generating function. The following assertions are equivalent: 
I. Ais a multiplier sequence. 
2. For all nonnegative integers n, the polynomial 
n on k 
r((+ 1) = (Past 
is real-rooted, and all its zeros have the same sign. 
3. Either Gp(x) or Gyx(—x) defines an entire function that can be written as 


Ga(+x) = Cx"e [T[c + Ox), 
k=], 


wheremEN, CER a0, & >Oforallk EN, and Ye) % < -. 


4. Ga(x) defines an entire function that is the limit, uniform on compact subsets 
of C, of real-rooted polynomials whose zeros all have the same sign. 


Example 7.7.2. Let A = {1/k!}{29. Then Ta((x+1)")) = Ln(—x), where Ly(x) is 
the nth Laguerre polynomial. Since orthogonal polynomials are real-rooted, we see 
that (2) of Theorem 7.7.1 is satisfied, and thus T is a multiplier sequence. 
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Only recently a complete characterization of linear operators preserving real- 
rootedness was obtained by Borcea and the author in [10]. This characterization is 
in terms of a natural extension of real-rootedness to several variables. A polynomial 
P(x1,---,Xm) € C[x1,...,Xm] is called stable if 


Im(x1) > 0,...,Im(%) > 0 implies P(x1,...,%) 40. 


By convention we also consider the identically zero polynomial to be stable. Hence a 
univariate real polynomial is stable if and only if it is real-rooted. Let a; <---< 
and B, <--- < By» be the zeros of two real-rooted polynomials. We say that these 
zeros interlace if 


0 <P) <m<Po<--- or Bi<a<fo<m<-:--. 


By convention, the “zeros” of any two polynomials of degree 0 or 1 interlace. In- 
terlacing zeros is characterized by a linear condition as the following theorem, often 
attributed to Obreschkoff, describes: 


Theorem 7.7.3 [72, Satz 5.2] Let f,g € R[x] \ {0}. Then the zeros of f and g inter- 
lace if and only if all polynomials in the linear space 


{af+Bg:a,B €R} 
are real-rooted. 


Let R, |x] = {p € Rix] : degp <n}. The symbol of a linear operator T : R, [x] > 
R{x] is the bivariate polynomial 
n n iS 
Gray) =T (ety) = ¥ (free Riss) 


k=0 


Theorem 7.7.4 [10] Let T : R,[x] > R[x] be a linear operator. Then T preserves 
real-rootedness if and only if (1), (2) or (3) below is satisfied. 


(1) T has rank at most two and is of the form 


T(p) =a(p)f+B(p)g, 


where 0, B : Rix] > R are linear functionals and f,g are real-rooted poly- 
nomials whose zeros interlace. 


(2) Gr(x,y) is stable. 
(3) Gr(x,—y) is stable. 


Example 7.7.5 The operators of type (1) are the ones achieved by Theorem 7.7.3. An 
example of an operator of type (2) is T = d/dx, because then Gr(x,y) =n(x+y)""1. 
An example of an operator of type (3) is the algebra automorphism, S : R[x] + R[x], 
defined by S(x) = —x. Indeed T is of type (2) if and only if T ©S is of type (3). 
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To illustrate how Theorem 7.7.4 may be used let us give a simple example from 
combinatorics. 


Example 7.7.6 The Eulerian polynomials satisfy the recursion An+\(x) = T,(An(x)), 
where 


T, =x(1 -2)5 +(n+1)x, 
see [94]. The symbol of T, : Ry{x| > R[x] is 
Tn ((x+y)") =x(Qxty)" "(e+ (n+ Dy +n), 


which is trivially stable. Hence A,(x) is real-rooted for all n € N by Theorem 7.7.4 
and induction. This was first proved by Frobenius [50]. 


A characterization of stable polynomials in two variables, and hence of the symbols 
of preservers of real-rootedness, follows from Helton and Vinnikov’s characteriza- 
tion of real-zero polynomials in [58], see [11]. 


Theorem 7.7.7 Let P(x,y) € R[x,y] be a polynomial of degree d. Then P is stable 
if and only if there exists three real symmetric d x d matrices A,B and C and a real 
number r such that 

P(x,y) =r-det(xA +yB+C), 


where A and B are positive semidefinite and A + B is the identity matrix. 


For the unbounded degree analog of Theorem 7.7.4 we define the symbol of a linear 
operator T : R[x] + R[x] to be the formal powers series 


ka T(x") 


Gr(xy) =T(e) = areca € RhlbI]). 


The Laguerre—Polya class, “—,,, is defined to be the class of real entire functions 
in n variables that are the uniform limits on compact subsets of C of real stable 
polynomials. For example exp(—x1x2 —x3x4 + 2x5) € Y-Ps since it is the limit of 
the stable polynomials 


(1 my" (1 my" (1 +253)", 

n n n 
Theorem 7.7.8 [10] Let T : R[x] > R[x] be a linear operator. Then T preserves 
real-rootedness if and only if (1), (2) or (3) below is satisfied. 


(1) T has rank at most two and is of the form 


T(p) =a(p)f+B(p)g, 


where a, B : R[x] > R are linear functionals and f,g are real-rooted polyno- 
mials whose zeros interlace. 


(2) Gr(x,y) € L-P». 
(3) Gr(x,-y) € G-Po. 
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There are, as of yet, no analogs of Theorems 7.7.4 and 7.7.8 for linear operators 
that preserve the property of having all zeros in a prescribed interval (other than R 
itself). 


Problem 7.7.9 Let I Cc R be an interval. Characterize all linear operators on poly- 
nomials that preserve the property of having all zeros in I. 


For polynomials appearing in combinatorics the case when J = (—co,0] is the 
most important. 


7.7.1 The subdivision operator 


An example of an operator of the kind appearing in Problem 7.7.9 is the “subdivi- 
sion” operator & : R[x] > R[x] in Section 7.3.2. The following theorem by Wagner 
proved the Neggers—Stanley conjecture for series-parallel posets, see [102, 103]. 


Theorem 7.7.10 [103] [fall zeros of €(f) and &(g) lie in the interval |—1,0], then 
so does the zeros of & (fg). 


As we have seen in Section 7.3.2, the next theorem has consequences in topological 
combinatorics. 


Theorem 7.7.11 [16] /f 


d 
F(x) = Vek (1 tx), 
k=0 
where hy > 0 for all0 <k < dz, then all zeros of & (f) are real, simple and located in 


The main part of the next theorem was proved by Brenti and Welker in [27], while 
(2) was proved in [41] . We take the opportunity to give alternative simple proofs 
below. 


Theorem 7.7.12 For each integer n > 2, € has a unique monic eigen-polynomial, 
Pn(x), of degree n. 
Moreover, 


(1) all zeros of p(x) are real, simple and lie in the interval |—1,0]; 


(2) pn(x) is symmetric around —1/2, i.e., 
(—1)"Pn(—1 —x) = pal). 


Proof. Let n > 2. Consider the map @ : [—1,0]" — [—1,0]” defined as follows. 
Let 0 = (61,...,O,) € [—1,0]". Since & preserves the property of having all zeros 
in [—1,0] (Theorems 7.7.10 and 7.7.11), we may order the zeros of the polynomial 
E((x— 01) ++ (x—O,)) as —1 <  <+++ << O& <0. Let 6(0) := (Q,...,Q,). By Hur- 
witz’ theorem on the continuity of zeros [82], @ is continuous. Hence by Brouwer’s 
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fixed point theorem @¢ has a fixed point, which then corresponds to a degree n eigen- 
polynomial, py, of &. It follows by examining the leading coefficients that the corre- 
sponding eigenvalue is n!. 

Set po := 1 and p; :=x+1/2. Let f be an arbitrary monic polynomial of degree 
n> 2, and let T =n!~'&. Now by expanding f as a linear combination of {pi} i_o» 


n 
f=Yapi, 
i=0 
we see that 


ee RS 
pet (f= pal (=) aiPi = Pn, 
since a, = 1. Hence p, is unique. By choosing f to be [—1,0]-rooted, we see that 
T*(f) is also [—1,0]-rooted for all k. By Hurwitz’ theorem, so is pn. Since pn is 
[—1,0]-rooted, it is certainly of the form displayed in Theorem 7.7.11. By Theorem 
7.7.11 again, the zeros of p, =n!~!&(p,) are distinct. 

Property (2) follows immediately from (7.9). | 


It is easy to see that the coefficients of p,(x) are rational numbers for each n > 2. 


Question 7.7.13 Is there a closed formula for py(x)? What are the generating func- 
tions 


AGH) = ¥ pats" iat Beg yy 


Note that &(B) =A. 


7.8 Common interleavers 


A powerful technique for proving that families of polynomials are real-rooted is that 
of compatible polynomials. This was employed by Chudnovsky and Seymour [35] 
to prove a conjecture of Hamidoune and Stanley on the zeros of independence poly- 
nomials of clawfree graphs. Subsequently an elegant alternative proof was given by 
Lass [63], by proving a Mehler formula for independence polynomials of clawfree 
graphs. An independent set in a finite and simple graph G = (V,E) is a set of pair- 
wise non-adjacent vertices. The independence polynomial of G is the polynomial 


1(G,x) = yo, 
Ss 


where the sum is over all independent sets $ C V. Recall that a claw is a graph 
isomorphic to the graph on V = {1,2,3,4} with edges F = {12, 13,14}. Note that 
the independence polynomial of a claw is 1 +4x+ 3x? +.x°, which has two non-real 
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zeros. A graph is clawfree if no induced subgraph is a claw. The next theorem was 
posed as a question by Hamidoune [57] and later as a conjecture by Stanley [92]. 


Theorem 7.8.1 [35, 63] If G is a clawfree graph, then all zeros of I(G,x) are real. 


Let f,g € R[x] be two real-rooted polynomials with positive leading coefficients. 
We say that f is an interleaver of g (written f < g) if 


+H < Po <a < fi, 


where {aj}"_, and {B;}", are the zeros of f and g, respectively. By convention 
we also write 0 << 0,0 < hand h <0, where h is any real-rooted polynomial with 
positive leading coefficient. If f < g and f £0, we say that f is a proper interleaver 
of g. The polynomials f\(x),..., fn(x) are K-compatible, where 1 < k < m, if 


Y Ajfi(z) 


jes 


is real-rooted whenever S C [m], |S| =k and A; > 0 for all j € S. The following 
theorem was used in Chudnovsky and Seymour’s proof of Theorem 7.8.1. 


Theorem 7.8.2 (Chudnovsky—Seymour [35]) Suppose that the leading coefficients 
of fi (x),---;fin(x) € Rix] are positive. The following are equivalent. 


1. fi(x),..-,fin(x) are 2-compatible; 

2. For alll <i< j<m, fi(x) and f;(x) have a proper common interleaver; 
3. fi(x),.--;fin(x) have a proper common interleaver; 

4. fi(x),..-; f(x) are m-compatible. 


Theorem 7.8.2 is useful in situations when the polynomials of interest may be 
expressed as nonnegative sums of similar polynomials. In order to prove that the 
polynomials of interest are real-rooted it then suffices to prove that the similar poly- 
nomials are 2-compatible. 

A sequence F, = (f;)/_, of real-rooted polynomials is called interlacing if f; << 
fj for all 1 <i < j <n. Let F, be the family of all interlacing sequences (f;)/_, of 
polynomials, and let .¥," be the family of (f;)?_, € Fn such that f; has nonnegative 
coefficients for all 1 <i <n. We are interested in when an m Xx n matrix G = (Gj;(x)) 
of polynomials maps -¥,, to Fy (or F," to F,") by the action 


n 
G- Fy = (81,---58m)’, where g, = Y° Gif; for all 1 <k <m. 
i=0 


i= 


This problem was considered by Fisk [46, Chapter 3], who proved some preliminary 
results. Since this approach has been proved succesful in combinatorial situations 
(see [86] where it was used to prove e.g. that the type D Eulerian polynomials are 
real-rooted), we take the opportunity to give a complete characterization for the case 
of nonnegative polynomials. 
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Lemma 7.8.3 If (f;)'_, and (gi)?_, are two interlacing sequences of polynomials, 
then the polynomial 


Sin + fogn-1 +++ + fn81 


is real-rooted. 


Proof. By Theorem 7.8.2 it suffices to prove that the sequence (fj%,41~i)/_, is 2- 
compatible. If i < j, then fjg,41—; is acommon interleaver of fign4,—; and fjgn41-;. 
Hence the lemma follows from Theorem 7.8.2. a 


See [86] for a proof of the following lemma. 


Lemma 7.8.4 Let f and g be two polynomials with nonnegative coefficients. Then 
f <g if and only if for all A, u > 0, the polynomial 


(Axt+u)f+s 
is real-rooted. 


Theorem 7.8.5 Let G = (G;;(x)) be an m x n matrix of polynomials. Then G: F,{ — 
F,* if and only if 


1. Gij has nonnegative coefficients for all i € [m] and j € [n], and 


2. forallAa,u>0,1<i<j<nandl<k<l<m 


(Ax + W)Gxj (x) + Goj(x) K (Ax + W) Gea (x) + Gei(x). (7.12) 
Proof. Let 
8k =) Guifi- 
i=0 


By Lemma 7.8.4, G: 4%," > F,, if and only if for all k < 2anda,p >0 


n 


(Ax+p)getge= )((Axt+H)Gu + Ga) fi = Yo Mnpiifi 
i=0 i=0 


is real-rooted and has nonnegative coefficients. The sufficiency follows from Lemma 
7.8.3, since if (7.12) holds, then the sequence (h;)_, is interlacing. To prove the 
necessity, let i< j and (f,)"_, be the interlacing sequence defined by 


1 if r =i, 
f(x)=(ax+pB ifr=j, and 
0 otherwise, 


where &,B > 0. Hence if G: ¥,. > F,7, then Mn41-; + (ax + B)hy+1_; is real- 


n 


rooted for all a, B > 0. Thus hy+1— 7 <hy+1-i, by Lemma 7.8.4, which is inequality 
(7.12). a 
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Corollary 7.8.6 Let G = (G;;) be an mx n matrix over R. Then G: F,5 > Ff if 
and only if G is TPa, i.e., all minors of G of size less than three are nonnegative. 


Proof. By Theorem 7.8.5 we may assume that all entries of G are nonnegative. 
Now 
(Ax + P)Gxj + Goji K (Axt+ UW) Gu + Go; 


for all A, us > 0 if and only if 
XGx; + Go; <K xGyj + Gy, 


which is seen to hold if and only if GyiGe; > GeiGz;. | 


If A = (Ay,A0,..-,Am) are integers such that 0 << Ay < Ag < +++ < Am <n, let 
Gz = (gi, (x)) be the m x n matrix with entries 


1 = x ifl<j<Aj;and 
ou Va ethene ise: 


The following corollary was first proved in [86]. 


Corollary 7.8.7 [If A = (A1,A2,...,Am) are integers such that0 <A, <An< +++ < 
An <n, then Gy: Fj) + Fr. 


Proof. — The possible 2 x 2 sub-matrices of Gy are 


x Xx x: 1 x | 1 1 1 1 d 1 1 

x xP Nx x Phe tlre we Phx 1 1 1 /]° 
By Theorem 7.8.5 we need to check (7.12) for these matrices. For example for the 
second matrix from the right we need to check 


(A+ 1)xtu«Kx(Ax+ut+1), 


for all A, 2 > 0, which is equivalent to 


+1 u 
< 
le 
which is certainly true. The other cases follows similarly. | 


Example 7.8.8 Let n be a positive integer and define polynomials Ay i(x), i € [n], by 


Anata) = xi 
o€Gn 
o(1)=i 
By conditioning on (2) =k, where o € Gy, and 6(1) =i, we see that 
Ansii(x) =) xAng(x) + Vo Ang(x), 1<isntt. 


k<i k>i 
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Hence if &, = (Anj(x))iL}, then 
+) = Go,1,2,...,n)* Lr- 


Since Ay = (1,x), we have by induction and Corollary 7.8.7 that &, is an interlacing 
sequence of polynomials for all n > 2. 


7.8.1 s-Eulerian polynomials 


Corollary 7.8.7 was used by Savage and Visontai [86] to prove real-rootedness of a 
large family of h*-polynomials. Let s = {s;}"_, be a sequence of positive integers. 
Define an integral polytope P; by 


Xx x x 
A= [Gio EROS 2 <2 cc Berl, 
S} $2 Sn 


The s-Eulerian polynomial may be defined as the h*-polynomial of P,: 


- : pees E;(x) 
» (Ps,k) (1—x)tl 


Savage and Schuster [85] provided a combinatorial description of s-Eulerian polyno- 
mials. The s-inversion sequences are defined by 


I, ={E = (e,...,€n) EN”: e;/s; < 1 forall 1 <i<n}. 
The ascent statistic on .% is defined as 
asc(&) = |{i € |[n] : ei1/si-1 < e:/s;}|, 
where & = (e1,...,€n), €9 =O and sy = 1. 


Theorem 7.8.9 [85] 


E(x) = y xase(é) 
ECs 


It turns out that several much studied families of polynomials in combinatorics are 
s-Eulerian polynomials for various s. For example the nth ordinary Eulerian poly- 
nomial corresponds to s = (1,2,...,”), while the nth Eulerian polynomial of type B 
corresponds to s = (2,4,...,2). If s = (s1,...,5n), let 


Exi(a) = Txt. 


E¢ Ig 


en=i 


It is not hard to see that the polynomials E, ;(x) satisfy the following recurrences, 
which make them ideal for an application of Corollary 7.8.7. 
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Lemma 7.8.10 [86] /fs = (s1,...,5n), 1 > 1, is a sequence of positive integers and 


0 <i<n, then 
ti-1 


Syed 
Es i(x) = 3 xEg j(x) + y Ey (x), 
j=0 Jeti 
where s' = (5s1,.--,8n—1) and t; = [isn—1/Sn]- 
An application of Corollary 7.8.7 proves the following theorem. 


Theorem 7.8.11 [86] fs = (s1,...,8,) is a sequence of positive integers, then E,(x) 


1 


is real-rooted. Moreover if n > 1, then the sequence {Egj(x)};"9_ is interlacing. 


7.8.2 Eulerian polynomials for finite Coxeter groups 


For undefined terminology on Coxeter groups we refer to [6]. Let (W,S) be a Coxeter 
system. The length of an element w € W is the smallest number k such that 


W = $1S2°--Sx, Wheres; € S forall 1 <i<n. 
Let €w(w) denote the length of w. The (right) descent set of w is 
Dyw(w) ={s eS: lw(ws) < w(w)}, 


and the descent number is desyw(w) = |Dw(w)|. The W-Eulerian polynomial of a 
finite Coxeter group W is the polynomial 


y desw (w) 


wew 


which is known to be the h-polynomial of the Coxeter complex associated to W, 
see [26]. The type A Eulerian polynomials are the common Eulerian polynomials. 
In [26], Brenti conjectured that the Eulerian polynomial of any finite Coxeter group 
is real-rooted. Brenti’s conjecture is true for type A and B Coxeter groups [26, 50], 
and one may check with the aid of the computer that the conjecture holds for the ex- 
ceptional groups H3, 4, Fy, E¢,E7, and Eg. Moreover, the Eulerian polynomial of the 
direct product of two finite Coxeter groups is the product of the Eulerian polynomials 
of the two groups. Hence it remains to prove Brenti’s conjecture for type D Coxeter 
groups. The type D case resisted many attempts, and it was not until very recently 
that the first sound proof was given by Savage and Visontai [86]. Their proof used 
compatibility arguments and ascent sequences. We will give a similar proof below 
that avoids the detour via ascent sequences. 

Recall that a combinatorial description of a rank n Coxeter group of type B is the 
group B,, of signed permutations o : [+n] — [+n], where [+n] = {+1,...,-tn}, such 
that o(—i) = —o(i) for all i € [+n]. An element o € B, is conveniently encoded 
by the window notation as a word 0; ---0,, where 0; = (i). The type B descent 
number of o is then 


desg(o) = |{i € |[n] : 5-1 > oF}, 
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where 69 := 0, see [26]. The nth type B Eulerian polynomial is thus 


B, (x) = y xAesa(0) | 


OCBn 


A combinatorial description of a rank n Coxeter group of type D is the group 
Dy, consisting of all elements of B, with an even number of negative entries in their 
window notation. The type D descent number of o € D, is then 


desp(o) = |{i € [n] : 5-1 > oF}, 


where 69 := — 02, see [26]. The nth type D Eulerian polynomial is 
Dn (x) = xdesp(o) : 


For n > 2 andk € [+n], let 


Dyx(x) = y xdesp(o) | 


o€Dn 
On=—k 


If k ¢ [n], we set D, ¢(x) :=0. The following table is conveniently generated by the 
recursion in Lemma 7.8.13 below. 


kK | Dox D3x% Dax 

—4/0 0 (x +1)? + 10x+1) 

—3/0 (x-+1)?  2x(x+1)(x+5) 

—2}|1 x(3+x) (3x? +14x+7) 

ae WIE: Qx(x-+1) x(5x* +14x+5) (7.13) 
lle Qx(x+1) x(5x*-+14x+5) 

2|x% = x(3xt+1) 2x(7x?-+14x+3) 

3 |0 x(x+1)? 2x(x+1)(5x+1) 

4/0 0O x(x+ 1) (x? + 10x +1) 


Note that the type D descents make sense for any element of B,, where n > 2. 


Lemma 7.8.12 [fn > 2, then 


Dag QO=—- YY. xe), (7.14) 
GEBn 
On=—k 


Proof. For k € [n}, let @ : By > Bn be the involution that swaps the letters k and 
—k in the window notation of the permutation. Clearly @ is a bijection between D, 
and B,, \ D, that preserves the type D descents for all n > 2. This proves (7.14) for 
k¢ {1,-1}. 

For k € [En], let B,[k] be the set of o € B, with o, =k. Then @, is a bijection 
between B,,[1] and B,,[—1] that preserves the type D descents for all n > 2. Similarly 
let D,,[k] be the set of o € D, with o, =k. Now B,[1] = D,[1]U ¢1(D,[—1]) and 
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B,|—1] = D,[—1] U ¢1(D,,[1]), where the unions are disjoint. Hence to prove (7.14) 
for k = +1, it remains to prove Dy,1(x) = Dy—1(x). We prove this by induction on 
n > 2, where the case n = 2 is easily checked. 

Consider the involution $24; : D,{1] > D,[—1], where n > 3. Then $24) pre- 
serves type D descents on o unless 0,1; = +2. Hence it remains to prove that 
the type D descent generating polynomials of D,,[2,1]U D,[—2,1] and D,[2,—1] U 
D,[—2, —1] agree, where D,,[k, ¢] is the set of o € D,, such that 0,_; = k and o, = @. 
This is easily seen by inspection. | 


Lemma 7.8.13 [fn > 2 andi € [+n], then 


Dn+1,i(%) = Yi xDn k(x) a "Dn x(x), ifi <Oand 
k<i k>i 

Dyiae) = Yi xDn k(x) +r Y" Dy x(x), ifi >0. 
k<i k>i 


Proof. |The lemma follows easily by using the alternative description (7.14) of 
Dy, i(x), and keeping track of 6,, where o € Dy+1[—i]. We leave the details to the 
reader. | 


Theorem 7.8.14 Let n> 2. The type D Eulerian polynomial D,,(x) is real-rooted. 
Moreover for each k € [+n], the polynomial D,, x(x) is real-rooted, and ifn > 4, 
then the sequence Dn := (Dnk(X)) ken) #8 interlacing. 


Proof. One may easily check that D,(x) and D,,;(x) are real-rooted whenever 2 < 
n<4andk € [+n], see (7.13). The sequence Yj is interlacing, see (7.13). By Lemma 
7.8.13, up to a relabeling of [+n], 


Dri = GanQn, 


where A” is a weakly increasing sequence. The matrix Gyn is of the type appearing 
in Corollary 7.8.7. Hence the theorem follows from Corollary 7.8.7. a 


Theorem 7.8.15 (Frobenius [50], Brenti [26], Savage—Visontai [86]) The Eulerian 
polynomial of any finite Coxeter group is real-rooted. 


Remark 7.8.16 For > 1 and i € [zn], define 


Bails) = xo, 


ocBn 
On=—i 


Then B,,; satisfies the same recursion as in Lemma 7.8.13, because the proof is ig- 
norant to what happens in the far left in the window notation of an element of By. 
Moreover this recursion is valid for all n > 1. Since (B;-1(x),B11(x)) = (1,x) is 
interlacing, induction and Corollary 7.8.7 implies that the sequence (Bn, i(x))ic[tn] is 
an interlacing sequence of polynomials for all n > 1. 
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7.9 Multivariate techniques 


To prove that a family of univariate polynomials are real-rooted it is sometimes easier 
to work with multivariate analogs of the polynomials. As alluded to in Section 7.7, 
a fruitful generalization of real-rootedness for multivariate polynomials is that of 
(real-) stable polynomials. There are several benefits in a multivariate approach; the 
proofs sometimes become more transparent, several powerful inequalities are avail- 
able for multivariate stable polynomials, it may give you a better understanding for 
the combinatorial problem at hand. An important class of stable polynomials are 
determinantal polynomials. 


Proposition 7.9.1 Let A),...,An be positive semidefinite hermitian matrices, and Ao 
a hermitian matrix. Then the polynomial 


P(x1, oes Xn) = det(Ao +x A} pees +XnAn) 
is either stable or identically zero. 


Proof. By Hurwitz’ theorem [33, Footnote 3, p. 96] and a standard approxima- 
tion argument we may assume that Aj is positive definite. Let x = (a) +ib1,...,dy+ 
ib,) € C" be such that a; € R and b; > 0 for all 1 < j <n. We need to prove that 
P(x) 40. Now P(x) = det(iB — A), where B = bjA, +--- + b,Azn is positive defi- 


nite and A = —Ap — ajA —---—ayAy is hermitian. Hence B has a square root and 
thus P(x) = det(B) det(iJ — B~'/2AB~'/?) 4 0, where / is the identity matrix, since 
B-'/24B7'/2 is hermitian and thus has real eigenvalues only. | 


For n = 2 a converse of Proposition 7.9.1 holds, see Theorem 7.7.7. The analog 
of Theorem 7.7.7 for n > 3 fails to be true by a simple count of parameters. For 
possible partial converses of Proposition 7.9.1, see the survey [100]. 

Recently attempts have been made to find appropriate multivariate analogs of fre- 
quently studied real-rooted univariate polynomials in combinatorics. Let us illustrate 
by describing a multivariate Eulerian polynomial. For o € G,, let 


DB(o) = {o(i): o(i— 1) > o(i)}, and 
AB(o) = {o(i): o(i— 1) < o(i)}, 


where o(0) = o(n +1) =o, be the set of descent bottoms and ascent bottoms of 
O, respectively. Let A, (x,y) be the polynomial in R[x1,...,%n,V1,---,Yn] defined by 


An(x,y) = y w(o), where w(o) = iE: I] »:- 


ocGy, i€DB(o) jE AB(o) 


For example w(573148926) = x5x3x|x2sy1yaygy2¥6. Generate a permutation o” in 
G, by inserting the letter 1 in a slot between two adjacent letters in a permutation 
000 +++ On_10y of {2,3,...,n} (where 09 = 0, =). Note that there is an obvious 
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one-to-one correspondence between the slots and the variables appearing in w(o’). 
Thus if we insert | in the slot corresponding to the variable z, then 


ae 
w(O) = x11 =-w(O 
(o) =my1z w(o) 
since the descent/ascent bottom corresponding to z in o’ will be destroyed, and 1 
becomes an ascent and descent bottom. We have proved 


Lars) fe) 
An xX, =x = eae An- Xs : ’ 
(x,y) =x (% Fs *,| 1(x*,y*) 


where x* = (x2,...,%,) and y* = (y2,...,¥n). To prove that A,(x,y) is stable for all 
n it remains to prove that the operators of the form )'\_, 0/dx; preserve stability. 
Stability preservers were recently characterized in [9]. The following theorem is the 
algebraic characterization. For « € N”, let Cx[x1,...,x,]| be the linear space of all 
polynomials that have degree at most k; in x; for each 1 <i <n. The symbol of a 
linear operator T : Cx[x1,.--,Xn] 3 C[x1,..-,Xm] is the polynomial 


Gr (X15 ++ Xm Vis-+-sYn) = T (1 +y1)" + On +n)", 
where T acts on the y-variables as if they were constants. 


Theorem 7.9.2 [9] Let T : Cx[x1,.--,Xn] 3 C[x1,.--,%m] be a linear operator of 
rank greater than one. Then T preserves stability if and only if Gr is stable. 


The symbol of the operator T = )}'i_ 0/0x; is 


n 
K; 
Gr = (xi tyi) +++ Gn tyn)™ — 
pars 


Hence if Im(x;) > 0 and Im(y;) > 0 for all 1 < j <n, then Im(x;+y;)~! <0, and 
hence the symbol is non-zero. Thus G7 is stable and by induction and Theorem 7.9.2, 
An(x,y) is stable for all n > 1. 

The multivariate Eulerian polynomials above and more general Eulerian-like 
polynomials were introduced in [22] and used to prove the Monotone Column Per- 
manent Conjecture of Haglund, Ono and Wagner [55]. Suppose A = (a; ae = isa 
real matrix that has weakly increasing down columns. Then the Monotone Column 
Permanent Conjecture stated that the permanent of the matrix (aj; +x)? j=1> Where 
x is a variable, is real-rooted. Subsequently multivariate Eulerian polynomials for 
colored permutations and various other models have been studied [23, 56, 101, 32]. 


7.9.1 Stable polynomials and matroids 


Let E be a finite set and let x = (x.)ecg be independent variables. The support of a 


multiaffine polynomial 
P(x) = }' a(S) [ ]e, 
SCE eS 
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is the set system Supp(P) = {S C E : a(S) £0}. Choe, Oxley, Sokal and Wagner [33] 
proved the following striking relationship between stable polynomials and matroids. 


Theorem 7.9.3 The support of a homogeneous, multiaffine and stable polynomial is 
the set of bases of a matroid. 


Hence Theorem 7.9.3 suggests an alternative way of representing matroids. A ma- 
troid M, with set of bases Z&, has the half-plane property (HPP) if its bases gener- 


ating polynomial 
Py(x) = ye [] 
BeBecB 
is stable, and M has the weak half-plane property (WHPP) if there are positive 
numbers a(B), B € &, such that 


y a(B) | [xe 


BEB e€B 


is stable. For example, the Fano matroid F7 is not WHPP, see [17]. The fact that 
graphic matroids are HPP is a consequence of the Matrix Tree theorem and Proposi- 
tion 7.9.1. Suppose V = [n], and let {6;}_, be the standard basis of R”. The weighted 
Laplacian of a connected graph G = (V,E) is defined as 


L(x) = y Xe (Se, a 8e,) (Ge, a Oes) 


ecE 


where e; and e2 are the vertices incident to e € E. We refer to [99, Theorem VI.29] 
for a proof of the next classical theorem that goes back to Kirchhoff and Maxwell. Let 
Tg(x) be the spanning tree polynomial of G, i.e., the bases generating polynomial 
of the graphical matroid associated to G. 


Theorem 7.9.4 (Matrix-tree theorem) For i € V, let Lg(x) ji be the matrix obtained 
by deleting the column and row indexed by i in Lg(x). Then 


Tg (x) = det (Lg (x)ii) . 


Clearly the matrices in the pencil Lg(x)j are positive semidefinite. Hence that 
graphic matroids are HPP follows from Theorem 7.9.4 and Proposition 7.9.1. A sim- 
ilar reasoning proves that all regular matroids are HPP, and that all matroids repre- 
sentable over C are WHPP, see [17, 33]. On the other hand, the Vamos cube Vs is 
not representable over any field, and still Vg is HPP [105]. For further results on the 
relationship between stable polynomials and matroids we refer to [17, 21, 33, 105]. 


7.9.2 Strong Rayleigh measures 


Stability implies several strong inequalities among the coefficients. Note that the 
multivariate Eulerian polynomial above is multiaffine, i.e., it is of degree at most one 
in each variable. We may view multiaffine polynomials with nonnegative coefficients 
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as discrete probability measures. If E is a finite set, x = (x-)ece are independent 


variables, and 
P(x) = ¥ a(S) [] xe, 
SCE ecS 


is a multiaffine polynomial with nonnegative coefficients normalized so that 
P(1,...1) = 1, we may define a discrete probability measure ps on 2” by setting 
U(S) = a(S) for each S € 2”. Then P, := P is the multivariate partition function of 
. A discrete probability measure yp is called strong Rayleigh if P,, is stable. Hence 
the measure HU, on 20"), defined by 


1 

Ln(S) = alte € G,: DB(o) Uf{i+n:i € AB(o)} =S}| 
is strong Rayleigh. A fundamental strong Rayleigh measure is the uniform spanning 
tree measure, [c, associated to a connected graph G = (V,E). This is the measure 

on 2° defined by 

1]1 if Sis a spanning tree, 
g(S) = ; 

0 otherwise 


where f is the number of spanning trees of G. The uniform spanning tree measures, 
and more generally the uniform measure on the set of bases of any HPP matroid, is 
strong Rayleigh by the discussion in Section 7.9.1. 

A general class of strong Rayleigh measures containing the uniform spanning 
tree measures is the class of determinantal measures, see [66]. Let C be a hermitian 
n Xn contraction matrix, i.e., a positive semidefinite matrix with all its eigenvalues 
located in the interval [0,1]. Define a probability measure on 2b] by 


Uc({T :T D> S}) =detC(S), for all SC [nl], 


where C(S) is the submatrix of C with rows and columns indexed by S. Using Propo- 
sition 7.9.1, it is not hard to prove that Uc is strong Rayleigh, see [12]. 

Negative dependence is an important notion in probability theory, statistics and 
statistical mechanics, see the survey [75]. In [12] several strong negative dependence 
properties of strong Rayleigh measures were established. Identify 2" with {0,1}. 
A probability measure UW on {0, 1}”" is negatively associated if 


J fedus f faw f sau, 


whenever f,g : {0,1}" — R are increasing functions depending on disjoint sets of 
variables, i.e., f(1) only depends on the variables n;,i € A, and g(1)) only depends on 
the variables 7;,i € B, where ANB =9. In particular setting f(7) = nj and g(n) = n7;. 
where i 4 j, we see that y is pairwise negatively correlated i.c., 


Hn: Mm =nj =1) Sun: mH =1)y(N: nj = 1). 
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Example 7.9.5 For n = 2, a discrete probability measure defined by (0) = 
a,u({1}) = b,u({2}) =c,u({1}) =, with a+b+c+d =1 is pairwise nega- 
tively correlated if and only if d(a+b+c+d) < (b+d)(c+d), i.e. if and only if 
ad < be. Also, it is easy to see that a real polynomial a+ bx, + cxz + dx x2 is sta- 
ble if and only if ad < be. By the next theorem the notions strong Rayleigh, negative 
association and pairwise negative correlation agree for n = 2. 


Theorem 7.9.6 [12] Jf tt is a discrete probability measure that is strong Rayleigh, 
then it is negatively associated. 


Recently Pemantle and Peres [77] proved general concentration inequalities for 
strong Rayleigh measures. A function f : {0,1}” + R is Lipschitz-1 if 


lf(n) — F(E)I < d(n,¢), for all Nis € {0, Les 
where d is the Hamming distance, d(n,§) = |{i€ |[n]: ni A &}1. 


Theorem 7.9.7 (Pemantle and Peres, [77]) Suppose yt is a probability measure on 
{0,1}" whose partition function is stable and has mean m = E(¥_, ni). If f is any 
Lipschitz-1 function on {0,1}", then 


u(n :|f(1) -Ef| > a) <5exp (aoa) ; 


7.9.3 The symmetric exclusion process 


The symmetric exclusion process (with creation and annihilation) is a Markov pro- 
cess that models particles jumping on a countable set of sites. Here we will just 
consider the case when we have a finite set of sites [n]. Given a symmetric matrix 
Q = (qi;); ;-1 of nonnegative numbers and vectors b = (b;)?_, and d = (d;)/_, of non- 
negative numbers, define a continuous time Markov process on {0,1}” as follows. 
Let n € {0, 1}” represent the configuration of the particles, with n; = 1 meaning that 
site i is occupied, and 1; = 0 that site i is vacant. Particles at occupied sites jump to 
vacant sites at specified rates. More precisely, these are the transitions in the Markov 
process, which we denote by SEP(Q,b,d), see Figure 7.4: 


(J) A particle jumps from site i to site j at rate q;;: The configuration 7) is un- 
changed unless n; = 1 and 1; = 0, and then nj; and 7; are exchanged in 17. 


(B) A particle at site i is created (is born) at rate b;: The configuration 7 is un- 
changed unless n; = 0, and then n; is changed from a zero to a one in 7). 


(D) A particle at site i is annihilated (dies) at rate dj: The configuration 77 is 
unchanged unless 7; = 1, and then nj; is changed from a one to a zero in 77. 


It was proved in [12, 104] that SEP(Q,b,d) preserves the family of strong 
Rayleigh measures. 
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dy by q21 q12 
qa} 44? 
413) q23 
a3 bs a3 434 
3. 4 


Figure 7.4 
The transitions in SEP(Q,b,d) on 4 sites, where gij = q ji. 


Theorem 7.9.8 [f the initial distribution of a symmetric exclusion process 
SEP(Q,b,d) is strong Rayleigh, then the distribution is strong Rayleigh for all posi- 
tive times. 


An immediate consequence of Theorem 7.9.8 is that the stationary distribution (if 
unique) of the symmetric exclusion process is strong Rayleigh. 


Corollary 7.9.9 If a symmetric exclusion process SEP(Q,b,d) is irreducible and 
positive recurrent, then the unique stationary distribution is strong Rayleigh. 


Proof. Choose an initial distribution that is strong Rayleigh. Then the partition 
function, P,(x), of the distribution at time t is stable for all t > 0, by Theorem 7.9.8. 
The partition function of the stationary distribution is given by lim,_,.. P,(x). By Hur- 
witz’ theorem [33, Footnote 3, p. 96] the partition function of the stationary distribu- 
tion is stable, i.e., the stationary distribution is strong Rayleigh. | 


In view of Corollary 7.9.9 it would be interesting to find the stationary distri- 
butions of SEP(Q,b,d) for specific parameters Q,b, and d. This was achieved by 
Corteel and Williams [36] for the parameters 


1 if|j—i]=1and 
dij = dip fe a8 
0 if|j—i>1, 


b=(a,0,...,0,8), (7.15) 
d=(y,0,...,0,B). 


Hence the particles jump on a line, where particles are only allowed to jump to neigh- 
boring sites, and be created and annihilated at the endpoints. The description of the 
stationary distribution is in terms of combinatorial objects called staircase tableaux. 
The special case when 6 = y = 0 is related to multivariate Eulerian polynomials. 
The excedence set, .2°(o) C [n], of a signed permutation o € B, was defined by 
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Steingrimsson [96] as 

|o(i)| >i, or; 

o(i) =—i. 


If o € By, let |o| € G, be the permutation where i++ |o(i)| for all 1 <i<n. A cycle 
c of |O| is called a negative cycle of o € B, if o(j) <0, where |o(j)| is the maximal 
element of c. Otherwise c is called a positive cycle of o. Let c_(o) and c,(o) be 
the number of negative and positive cycles of o, respectively. 


i¢ &(o) if and only if 


Theorem 7.9.10 [23] The multivariate partition function of the symmetric exclusion 
process on 2l"| with parameters as in (7.15), with 6 = y = 0, is a constant multiple 


of 
2 c_(o) ae 
& as Xj. (7.16) 
», @ B ale 


Note that by Corollary 7.9.9, the polynomial (7.16) is stable. 


Problem 7.9.11 Find the stationary distribution of SEP(Q,b,d) for parameters 
other than (7.15). 


7.9.4 The Grace—Walsh-Szeg6 theorem, and the proof of 
Theorem 7.5.1 


The proof of Theorem 7.5.1 is an excellent example of how multivariate techniques 
may be used to prove statements about the zeros of univariate polynomials. The proof 
uses a combinatorial symmetric function identity and the Grace—Walsh—Szegé6 theo- 
rem, which is undoubtedly one of the most useful theorems governing the location 
of zeros of polynomials, see [82]. 

A circular region is a proper subset of the complex plane that is bounded by 
either a circle or a straight line, and is either open or closed. 


Theorem 7.9.12 (Grace—Walsh-Szeg6) Let f © Cz1,...,Zn] be a multiaffine and 
symmetric polynomial, and let K be a circular region. Assume that either K is convex 
or that the degree of f isn. For any (,...,n € K there is a number € € K such that 


F(G15---1n) = f(G,---56). 


The second ingredient in the proof of Theorem 7.5.1 is the following symmetric 
function identity. Let e,(x) be the kth elementary symmetric function in the variables 
Ki Rh) 


Lemma 7.9.13 For nonnegative integers n, 
n , [n/2| 1 
Y (eux)? —exs(x)ensa(s)) =en(s) Yo Ceeran (xt), ID 
k=0 k=0 


x+1/x = (x1 +1/x1,...,% +1/%n) and CG = EY /(e+ 1), k EN, are the Catalan 
numbers. 
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Proof. — For undefined symmetric function terminology, we refer to [93, Chapter 7]. 
The polynomial e,(x)? — eg_1(x)ex41(x) is the Schur-function s,.(x), where 2* = 
(2,2,...,2). We may rewrite (7.17) as 


n [2/2| 
Y sx) = YG Y [uf ]at). (7.18) 
= k= |S|=2kieS jd 


By the combinatorial definition of the Schur-function, the left-hand side of (7.18) 
is the generating polynomial of all semi-standard Young tableaux with entries in 
{1,...,n}, that are of shape 2* for some k € N. Call this set %. Given T € %, let 
S be the set of entries that occur only once in T. By deleting the remaining entries 
we obtain a standard Young tableau of shape 2", where 2k = |S|. There are exactly 
Cy standard Young tableaux of shape 2* with set of entries S, see e.g. [93, Exercise 
6.19.ww]. It is not hard to see that the original semi-standard Young tableau is then 
determined by the set of duplicates. This explains the right-hand side of (7.18). 


Proof of Theorem 7.5.1. Let P(x) = Li_o ayx* = T]_o(1 + pxx), where py > 0 for 
all 1 <k <n, and let 


n 
O(x) = V (ag — ax—ran41)x*. 
k=0 
Suppose there is a number € € C, with ¢ ¢ {x © R: x < 0}, for which Q(¢) =0. We 
may write € as € = E?, where Re(&) > 0. By (7.17), 


lw : ‘ 
0 = O(C) = an » Cr€n—2k (ng + DET Pa 4 =5) ; 


Since Re(pj;§ +1/(pj;§)) > 0 for all 1 < j <n, the Grace—Walsh-Szegé Theorem 
provides a number 7 € C, with Re(7) > 0, such that 


Ln/2 oo ce ; 
0= Cren—2k (15-51) = ca )n" =: 1"qn (=z) : 
k=0 k=0 2k 1? 


Since Re(1) > 0, we have 1/n? € C\ {x € R: x < 0}. Hence, the desired contradic- 
tion follows if we can prove that all the zeros of the polynomial g,(x) are real and 
negative. This follows from the identity 


k=0 k=0 
1 (1) (14+x 
=a "Pr (=). 


where {POV}, are Jacobi polynomials, see [83, p. 254]. The zeros of the Ja- 


cobi polynomials (PO) (x) },, are located in the interval (—1,1). Note that the first 
identity in the equation above follows immediately from (7.17). | 
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7.10 Historical notes 


Here are some complementary historical notes about the origin of some of the central 
notions of this chapter. 

Although some combinatorial polynomials such as the Eulerian polynomials 
have been known to be y-positive for at least 45 years [48], Gal [51] and the au- 
thor [15] realized the relevance of y-positivity to topological combinatorics and in 
particular to the Charney—Davis conjecture. 

Multivariate stable polynomials and similar classes of polynomials have been 
studied in many different areas. For their importance in control theory, see [45] and 
the references therein. In statistical mechanics they play an important role in Lee and 
Yang’s approach to the study of phase transitions [64, 106]. In PDE theory so-called 
hyperbolic polynomials play an important part in the existence of a fundamental 
solution to a linear PDE with constant coefficients, see [60]. The importance of stable 
polynomials in matroid theory was first realized in [33]. An important application of 
stable polynomials to a problem in combinatorics is the proof by Gurvits of a vast 
generalization of the Van der Waerden conjecture, [54]. A recent application is the 
spectacular solution to the Kadison—Singer problem by Marcus et al. [67]. See the 
surveys [76, 104] for further applications of stable polynomials. 

The notion of HPP and WHPP matroids were introduced in Choe et al. [33]. The 
strong Rayleigh property was introduced for matroids by Choe and Wagner [34], and 
extended to discrete probability measures and studied extensively in [12]. 
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8.1 Introduction 


Combinatorics on words is a field that has both historical roots and a substantial 
growth. Its roots are to be found in the early results of Axel Thue on square free 
words and the development of combinatorial group theory (see [4] for an introduction 
to the early developments of combinatorics on words). The present interest in the 
field is pushed by its links with several connections with other topics external to pure 
mathematics, notably bioinformatics. 

Enumerative combinatorics on words is itself a branch of enumerative combi- 
natorics, centered on the simplest structure constructor since words are the same as 
finite sequences. 

In this chapter, we have tried to cover a variety of aspects of enumerative combi- 
natorics on words. We have focused on the problems of enumeration connected with 
conjugacy classes. This includes many interesting combinatorial aspects of words 
like Lyndon words and de Bruijn cycles. One of the highlights of the chapter is 
the connection between both of these concepts via the theorem of Fredericksen and 
Maiorana. 

We have put aside some important aspects of enumerative combinatorics on 
words, which would deserve another complete chapter. This includes the enumer- 
ation of various families of words subject to a restriction. For example, the enumera- 
tion of square-free words is an important problem for which only asymptotic results 
are known. It is known for example that the number s,, of ternary square-free words 


of length n satisfies lim,_4.. sil = 1.302... (see [39] or [16]). Other examples of 
interest include unbordered words or words avoiding more general patterns (on this 
notion, see [31]). 

The chapter is organized as follows. 

In Section 8.2, we introduce some basic definitions concerning words used in 
subsequent sections. We also introduce basic notions concerning generating series 
and automata. Both are powerful tools for the enumeration of words. 

In Section 8.3, we introduce the notion of conjugacy and the correlated notions 
of necklaces or circular codes. These notions play a role in almost all the remaining 
sections of the chapter. We review some classical formulas such as Witt’s Formula or 
Manning’s Formula for the zeta function of a set of words. 

In Section 8.4, we introduce Lyndon words and prove the important factorization 
theorem (Theorem 8.4.3). We also discuss the problem of generating Lyndon words 
and present algorithms for generating them in alphabetic order. 

In Section 8.5 we introduce the notion of de Bruijn cycle and their relation with 
Eulerian graphs. We prove the so-called BEST theorem, enumerating the spanning 
trees in an Eulerian graph, and apply it to the enumeration of de Bruijn cycles. We 
finally present the theorem of Fredericksen and Maiorana [17] which beautifully 
connects Lyndon words and de Bruijn cycles (Theorem 8.5.8). 

In Section 8.6, we introduce unavoidable sets. We prove that, on any alphabet, 
there exist unavoidable sets of words of length n, which are a set of representatives 
of the conjugacy classes of words of length n (Theorem 8.6.11). 
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In Section 8.7, we introduce a transformation on words, known as the Burrows— 
Wheeler transformation. This transformation is used in text compression. It is closely 
related with conjugacy. 

We show in Section 8.8 that the Burrows—Wheeler transformation is closely re- 
lated with a well-known bijection on words, known as the Gessel-Reutenauer bijec- 
tion. We also prove some results due to Higgins [23], which generalize the theorem 
of Fredericksen and Maiorana (Theorem 8.8.9). 

In Section 8.9, we show that the Burrows—Wheeler transformation is also related 
to a well-known concept in string processing, the so-called suffix arrays. We end the 
section with several results due to Schurman and Stoye [38] concerning the enumer- 
ation of suffix arrays. 


8.2 Preliminaries 


We briefly introduce the basic terminology on words. Let A be a finite set usually 
called the alphabet. The elements of A are called letters. 

A word w on the alphabet A is denoted w = a) a2---a, with a; € A. The integer 7 is 
the length of w. We denote as usual by A* the set of words over A and by € the empty 
word. For a word w, we denote by |w| the length of w. We use the notation At = 
A* — {€}. The set A* is a monoid. Indeed, the concatenation of words is associative, 
and the empty word is a neutral element for concatenation. The set A* is sometimes 
called the free semigroup over A, while A* is called the free monoid. 

A word w is called a factor (respectively, a prefix, respectively, a suffix) of a 
word u if there exist words x,y such that u = xwy (respectively, u = wy, respectively, 
u = xw). The factor (respectively, the prefix, respectively, the suffix) is proper if 
xy # € (respectively, y 4 €, respectively, x  €). The prefix of length k of a word w 
is also denoted by w[0..k — 1]. 

The set of words over a finite alphabet A can be conveniently seen as a tree. 
Figure 8.1 represents the set {a,b}* as a binary tree. The vertices are the elements of 
A*. The root is the empty word €. The sons of a node x are the words xa for a € A. 
Every word x can also be viewed as the path leading from the root to the node x. A 
word x is a prefix of a word y if it is an ancestor in the tree. Given two words x and 
y, the longest common prefix of x and y is the nearest common ancestor of x and y in 
the tree. 

The set of factors of a word x is denoted F(x). We denote by F(X) the set of 
factors of words in a set X C A’. 

The lexicographic order, also called alphabetic order, is defined as follows. 
Given two words x,y, we have x < y if x is a proper prefix of y or if there exist 
factorizations x = uax’ and y = uby’ with a,b letters and a < b. This is the usual 
order in a dictionary. 


488 Handbook of Enumerative Combinatorics 


a" 
/ /. 
AAAK 


aaa aab aba abb baa bab bba 


Figure 8.1 
The tree of the free monoid on two letters. 


A border of a word w is a nonempty word that is both a prefix and a suffix of w. 
A word w is unbordered if its only border is w itself. For example, a is a border of 
aba and aabab is unbordered. 


8.2.1 Generating series 


For a set X of words, we denote by fx(z) = Y,s9Card(X A”)z” the generating 
series of X. 

Operations on sets can be transferred to their generating series. First, if X,Y are 
disjoint, then 


fxuy (z) = fx(z) + fr (z)- (8.1) 


Next, the product XY of two sets X,Y is defined by XY = {xy |xeX,y © Y}. We say 
the the product is unambiguous if xy = x’y’ for x,x’ € X and y,y’ € Y implies x =’ 
and y = y’. Then if the product of X,Y is unambiguous 


Fy (z) = Fx (z) fr (2). (8.2) 


A set X C A® is a code if the factorization of a word in words of X is unique. 
Formally, X is a code if x1x2---%n = y1y2°+*Ym With x;,y; € X and n,m > | implies 
n=mandx;=y;forl <i<n. 

As a particular case, a prefix code is a set that does not contain any proper prefix 
of one of its elements. The submonoid generated by a prefix code X is right unitary, 
that is to say that u, uv € X* implies v € X*. Conversely, any right unitary submonoid 
is generated by a prefix code. 

If X is a code, then , 


ise @). (8.3) 


fx«(z) = 
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In fact, since the sets X",X™ are disjoint for n #4 m, we have fy+(z) = 
Ln>o fxx(z). By unique decomposition, we also have fxn(z) = (fx(z))”. Thus 
f(z) = Lnso fx (z)” whence the result. 


Example 8.2.1 Let X = {a,ba}. The set X is a prefix code. We have Card(X* NA") = 
(<): Indeed, a word in X‘ A" is a product of n—k words ba and 2k —n words a. 
It is determined by the choice of the positions of the n—k words ba among k possible 
ones. 

On the other hand, Card(X* NA") = F,+1 where F, is the Fibonacci sequence 
defined by Fy = 0, Fi = 1 and Fy4, = Fy + Fy_1 for n = | (the first values are given 
in Table 8.1). 


6 7 8 9 10 11 12 13 
8 13 21 34 55 89 144 233 


Table 8.1 
The first values of the Fibonacci sequence. 


This is a consequence of the fact that fx+(z) = —_, by Equation (8.3). Since 


1—z—-z 
fx*(z) = Lecco fe (Z) we obtain the well-known identity relating Fibonacci numbers 


and binomial coefficients 
k 
Fus1 = s: es (8.4) 


k<n 


which sums binomial coefficients along the parallels to the first diagonal in Pascal’s 
triangle (see Table 8.2). 


1 
ak a 
i 9 oa, 2 
1 oF 9h B.S - 
ae en ae Ge Se a | 
R98. Sy Os 107 Sr 4 
5 16 15 20 15 6 1 


Table 8.2 
Pascal’s triangle. 


Example 8.2.2 The Dyck set is the set of words on the alphabet {a,b} having an 
equal number of occurrences of a and b. It is a right unitary submonoid and thus it 
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is generated by a prefix code D called the Dyck code. Let D, (respectively, Dp) be 
the set of words of D beginning with a (respectively, b). We have 


Dg=aDib and D, =bDja. (8.5) 


Let us verify the first one. The second one is symmetrical. Clearly any d € Dg ends 
with b. Set d = ayb. Then y has the same number of occurrences of a and b and thus 
y € D*. Set y=y1-+-yn with y; € D. If some y; begins with b, then ay,-+-yi—\b is a 
proper prefix of d that belongs to D*, a contradiction with the fact that D is a prefix 
code. Thus all y; are in Dg and y € aD%b. Conversely, any word in aD%b is clearly in 
Da. 

Since all products in (8.5) are unambiguous, we obtain fp,(z) = 2? fps (z). Since 
Dag is a code, by (8.3), this implies fp,(z) = 2/(1— fp, (z)). We conclude that fp, (z) 
is the solution of the equation 


y(z)’ —y(z) +2? =0. (8.6) 
such that y(0) = 0. Thus, we obtain the formula 


(fia 
fo, (2) = ae (8.7) 


Finally, since D = Dg UD, and fp,(z) = fp, (z) for reasons of symmetry, we obtain 


fo(z) =1-—vV 1-422 (8.8) 


Using the binomial formula, we obtain Card(DNA2") = —(—4)" (2); An elementary 
computation shows that (1/2) = (2(-1)""! /n4") 2-7). Thus 


n 


Card(DNA2") = ; c Z . (8.9) 


n—-1 


As a consequence, and since Dg = aD%b by (8.5), we obtain the important and well- 
known fact that 


[sf 
Card(D.NA*) = —— ( . (8.10) 
n n 


These numbers are called the Catalan numbers (see Table 8.3). 


4 5 6 7 8 9 10 
5 14 42 132 429 1430 4862 


Table 8.3 
The first Catalan numbers. 
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8.2.2. Automata 


An automaton on the alphabet A is given by a set Q of states,a setE CQxAx@Q 
of edges, a set J of initial states and a set T of terminal states. The automaton is 
denoted < = (QO,E,1,T) or (Q,1,T) if E is understood. 


Figure 8.2 
An automaton. 


Example 8.2.3 Figure 8.2 represents an automaton with two states and three edges. 
The initial edges are indicated with an incoming edge and the terminal ones with 
with an outgoing edge. Here state | is both the unique initial and terminal state. 


A path in the automaton is a sequence of consecutive edges (p;,q;, pi41) for 1 <i< 
n. The integer n is the length of the path. The word w = aja2---dy is its label. We 
denote p, —> py such a path. A path i “>t is successful if i ¢ J and t € T. The set 
recognized by the automaton is the set of labels of successful paths. The automaton 
is said to be unambiguous if for each word w there is at most one successful path 
labeled w. Thus, an unambiguous automaton defines a bijection between the set of 
successful paths and the set of their labels. As a particular case, an automaton is 
deterministic if it has at most one initial state and for each state p, at most one edge 
labeled by a given letter starting at p. 


Example 8.2.4 The automaton represented in Figure 8.2 recognizes the set {a, ba}* 
of Example 8.2.1. It is deterministic and thus unambiguous. 


The adjacency matrix of the automaton < = (Q,E,/,T) is the Q x Q-matrix with 
integer coefficients defined by 


Mp q = Card{e € E | e = (p,a,q) for some a € A}. 


It is clear that for each n > 1, M7, is the number of paths of length n from p to q. 


Thus we have the following useful statement. 


Proposition 8.2.5 Let < = (Q,1,T) be an unambiguous automaton, let M be its 
adjacency matrix and let X be the set recognized by &. For eachn > 1, 


Card(XNA")= Yo Mi,. 
ielteET 
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Example 8.2.6 The adjacency matrix of the automaton represented in Figure 8.2 is 


It is easy to verify that, forn > 1, 


F, F, 
M= n+1 nye 
| Fy Fr-1 


Thus, by Proposition 8.2.5, we have Card({a,ba}* NA") = Fy41, as already seen in 
Example 8.2.1. 


8.3. Conjugacy 


We define necklaces and primitive necklaces. We enumerate first primitive neck- 
laces (Witt’s Formula, Proposition 8.3.4) and then arbitrary ones (Proposition 8.3.6). 
See [30] for a more detailed presentation. These notions have been extended to more 
general structures (see in particular the case of partial words in [6]). 


8.3.1 Periods 


An integer p > 1 is a period of a word w = aja2---a, where a; € A if a; = aj+p for 
i=1,...,n—p. The smallest period of w is called the minimal period of w. 


Proposition 8.3.1 (Fine, Wilf) /f p,q are periods of a word w of length > p+q—- 
gced(p,q), then w has period gcd(p,q). 


Proof. Set w = a1a2---dy, with a; € A and d = gcd(p,q). We may assume that 
p = q. Assume first that d = 1. Let us show that p — q is a period of w. Let i be such 
that] <i<n—p+q.Ifi<n—p, we have a; = 4j4p = 4j+p—q. Otherwise, we have 
i>n-— pand thusi > q—1. Then a; = aj, = dj+p—q. Thus w has period p — gq. Since 
gcd(p,q) = gcd(p — q,q) we obtain by induction on p + q that w has period 1. 

In the general case, we consider the alphabet B = A%. On this alphabet w has 
periods p/d, g/d and length n/d > p/d+q/d. By the first part, it has period 1 as a 
word on the alphabet B and thus period d on the alphabet A. a 


Example 8.3.2. The word w = abaababaaba has periods 5 and 8 and length 11 = 
5+ 8-2. By Proposition 8.3.1, no word of length 12 can have periods 5 and 8 
without having period 1. 

More generally, let x, be the Fibonacci sequence of words defined by x, = b, 
Xp =a and Xn44 = XpXp_) forn => 2. For n > 3, let y, be the word x, minus its two 
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last letters. The word y7 is the word w above. Then, for n = 6, yy+1 has periods F,, 
and F,_,. Indeed, Yn+1 = %nYn—1 Shows that yy+1 has period F,. Moreover, 


Ynt+l = XnYn-1 = Xn—-1Xn—-2Xn—2Yn—-3 = Xn—1Xn—-2Xn—3Xn—4Yn—-3 


= Xn-1Xn—-1Xn—4Yn—-3 


which shows that F,—, is a period since Xn—4Yn—3 is a prefix of Xn—3 and thus of Xn—1. 
Since |yn41| = Fn + Fr-1 — 2, this shows that the bound of Proposition 8.3.1 is the 
best possible. 

A word w € A® is primitive if w =u" for u € A* implies n= 1. 

Two words x, y are conjugate if there exist words u,v such that x = uv and y = vu. 
Thus conjugate words are just cyclic shifts of one another. Conjugacy is thus an 
equivalence relation. The conjugacy class of a word of length n and period p has 
p elements if p divides n and has n elements otherwise. In particular, we note the 
following result. 


Proposition 8.3.3 A primitive word of length n has n distinct conjugates. 


8.3.2 Necklaces 


A class of conjugacy is often called a necklace, represented on a circle (read clock- 
wise, see Figure 8.3). 


OOOO 
Figure 83 


The six primitive necklaces of length 5 on the alphabet {a,b}. 


Let p(n,k) be the number of primitive necklaces of length n on k letters. Every 
word of length n is in a unique way a power of a primitive word of length d with d 
dividing n and such a word has d distinct conjugates. Thus, for any n > 1, 


k" =\"d p(d,k). (8.11) 
d\n 
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This can be written, using generating series, as a formula called the Cyclotomic 


Identity. 
1 1 


l-k AS 2 pn 


Indeed, taking the logarithm of both sides in Equation (8.12), we obtain 


(8.12) 


kitzn 
Y —p(n,k) log(1 —z") 
n>1 i n>1 
zim n 
n>1 m>| 1 n>1n=de 


and thus k"/n = Y,—ge p(d,k)/e whence Formula (8.11). 
We are going to find a converse giving an expression for the numbers p(n,k). 
This solution of the system of linear equations (8.11) uses the following function. 
The Mobius function is defined by (1) = 1 and forn > 1 


ie) (—1)! if nis the product of i distinct prime numbers 
n)= 
. 0 otherwise 


8 9 10 
1 -! 0 -1 1 -1 00 1 


Table 8.4 
The first few values of the MGbius function. 


Proposition 8.3.4 (Witt’s Formula) The number of primitive necklaces of length n 
on k letters is p(n,k) = Vain (n/d)k4. 


Table 8.5 gives the first few values of p(n,k). We prove some properties of the 
Mobius function before giving the proof of Proposition 8.3.4. 


Proposition 8.3.5 One has 


Ea = {0 pot 


d\n 0 otherwise 


Proof. Indeed, for n > 2, letn = pi! . +» pk and d = pi! .. p'm be the prime decom- 
positions of n,d. Then y(d) # 0 if and only if all ¢; are 0,1 and then (d) = (—1)' 
with t = y” | £;. Moreover, there are a possible choices giving the same sum f. 


Thus 
Fal) =Y-1)(") =0 


d|n f= 


since, by the binomial identity, the last expression is (1 — 1)”. | 
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n 1 2 3 4 5 6 7 8 9 
patito 0 0. © 0 © 0 0 
puD) (2b dn <B> “6 O° 18° ZO 
pin,3)|3 3 8 18 48 116 312 
pin,4)|4 6 20 60 204 670 

p(n,5)|5 10 40 150 476 

pu) "6 15-30" 195 

p(n,7)|7 21 27 

p(n,8) | 8 28 

p(n,9) | 9 


Table 8.5 
The number p(n,k) of primitive necklaces of length n on k letters for2 <<k+n< 10. 


For two functions a, B from N \0 into a ring R, their convolution product is the 
function a « B : N\ 0 — R defined by 


a*B(n)= ) a(d)B(e). 


de=n 


This product is associative with neutral element the function 1 with value 1 on 1 
and 0 elsewhere. By Proposition 8.3.5 the function n+ Yiajin L(d) is the function 1. 
This shows that the Mobius function is the inverse for the convolution product of the 
constant function equal to 1. 


Proof of Proposition 8.3.4. Set a(n) = k" and B(n) = np(n,k). Since k” = 
Ya\jndp(d,k) by Equation (8.11), we have @ = B * y where y is the constant func- 
tion equal to 1. Since y* u = 1, the convolution product of both sides by the Mobius 
function gives a@ *« LU = B, that isnp(n,k) = Yn—ge U(d)KE. | 


Recall that Euler’s totient function @ is defined as follows. The value of @(n) 
for n > 1 is the number of integers k with 1 <k <n such that gcd(n,k) = 1. See 
Table 8.6 for the first few values of this function. In other words, for n > 2, p(n) is 
the number of integers k for | < k <n that are prime to n. One has n = Yiyiy, o(d). 


ne Ald 2 
1 


4 
o(n) | 1 2 


3 5 6 7 8 9 10 
2 42646 4 
Table 8.6 

The values of the Euler function @(n) for n < 10. 


Indeed, for each divisor d of n the set My of integers m <n such that ged(n,m) = d 
has p(n/d) elements. Thus n = Y'4\, Card(Ma) = Vain P(2/d) = Lajn P(d). 
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) 


5 6 7 8 9 
1 1 1 1 1 
8 14 20 36 
11 24 51 130 315 

10 24 70 208 700 

481 


OMDNADNDNPWN KH] Re 
— 
Nn 
BK 
Nn 
= 
on 
Nn 


On Ye OOO. OD. Oe OOS 
sss 8 8 885858 
OMDNDNFPWN KH 


YS Ss SS SA AS SA 


Table 8.7 
The values of the number c(,k) of necklaces of length n on k letters for 2 <<k+n< 
10. 


Let c(n,k) be the number of necklaces of length n on k letters. Table 8.7 gives the 
first values of the numbers c(n,k). The values in Table 8.7 can be easily computed 
from those of Table 8.5 using the fact that c(n,k) = Yajn p(d,k). The following state- 
ment gives a direct way to compute the numbers c(n,k) (see [21], where it is credited 
to McMahon). 


Proposition 8.3.6 The equality c(n,k) = Ly ain e(n/d)k4 holds. 


Proof. — Consider the multiset formed by the n circular shifts of the words of length 
n (each word of length n may appear several times). The total number of the shifts 
in nc(n,k). On the other hand, each word w = ag---dy_1 of length n appears with 
a multiplicity, which is the number of integers p with 0 < p <n such that w = 
Gp***An—140***Ap—1, that is which are a period of w*. But Pp is a period of w? if and 
only if w is a power of a word of length gcd(n, p). Thus 


ne(n,k) = Yi Keed), (8.13) 


O<p<n 


Since there are g(n/d) integers p with 0 < p <n such that d = gcd(n, p), the result 
follows. a 


We illustrate the proof of Proposition 8.3.6 in the following example. 


Example 8.3.7 Let A = {a,b}. The multiset of circular shifts of words of length 4 is 
the multiset of 6 x 4 = 24 elements represented below. 


aaaa aaaa aaaa aaaa 
aaab aaba abaa_ baaa 
aabb abba bbaa_ baab 
abab baba abab_ baba 
abbb babb bbab_ bbba 
bbbb bbbb bbbb  bbbb 
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The words appearing more than once are abab,baba, which appear twice, and 
aaaa, bbbb, which appear four times. 

The following array gives for each value of p = 1,2,3 the set of words w of length 
4 such that p is a period of w* (for p =0 it is the set of all words of length 4). 


P ged(p,4) 
0 | aaaa, aaab, aaba, aabb, abaa, abab, abba, abbb, 
baaa, baab, baba, babb, bbaa, bbab, bbba, bbbb | 4 
1 | aaaa,bbbb 1 
2 | aaaa,abab,baba,bbbb 2 
3 | aaaa,bbbb 1 


The value of d = gcd(p,4) is indicated on the right. The corresponding prefix of 
length d of each word is indicated in boldface. The row indexed p contains 24 ele- 
ments coresponding to the binary words of length d in boldface. In this way we have 
illustrated Equation (8.13) since summing the cardinalities of the sets in each row, 
we obtain 24 = 16+2+4+2. 


8.3.3. Circular codes 


A circular code is a set of words X on the alphabet A such that any necklace has a 
unique factorization in words of X. In particular, a circular code is a code. 

Formally, X is a circular code if for x1,...,x%, and y1,...,¥ in X the equality 
§X2°**XyP =J1-*+Ym With x; = ps and s nonempty implies n = m, p = 1 and x; = y; 
for 1 <i<n. 


Example 8.3.8 The set X = {a,ba} is a circular code. Indeed, there is at most one 
way to paste every occurrence of b with the a following it. 


Example 8.3.9 The set X = {ab, ba} is not a circular code. Indeed, the necklace of 
ab has two possible factorizations. 


It can be shown that a submonoid M of A* is generated by a circular code if and 
only if it satisfies the following condition for any u,v € A*: 


uv, ue MSu,veM. (8.14) 
For a proof, see [5, Chapter 7]. Note that (8.14) implies for any u € M andn > 1 
uEMSucM. (8.15) 


Let S be a set of words on the alphabet A and let s,, = Card(SMA”) in such a way 


that f(z) = Ynso5n2". 
The zeta function of S is the series 


Ss 
Cs(z) =exp y oh. 
n>1 n 


The following is due to Manning (see [5, Chapter 7]). The proof given below uses 
an argument due to [41]. 
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Theorem 8.3.10 Let X be a circular code and let S be the set of words having a 
conjugate in X*. Then 


1 
€s(z) = 1— fx(z)’ (8.16) 
or equivalently flo) 
— £IXK 
fs(z) = Tofe@) (8.17) 


Proof. For x € X, denote g,, the number of words of the form w = syp of length 
n with y € X* and x = ps with p nonempty. Since X is circular, the triple (s,y, p) is 
uniquely determined by w. Conversely, every word of SMA” is of this form for some 
x EX. Thus gyn = |x|Card(X* A") and Card(SNA") = Yxex gn.x- We obtain 


Card(SMA") = Y gnx= Y- |x|Card(x* NA" P') 
xEX xEX 
n 
= ) mCard(X NA”) Card(X* NA"). 


m=0 


This shows that fs(z) = zfy (z),fx«(z) whence Formula (8.17). Formula (8.16) is ob- 
tained from (8.17) by taking the derivative of the logarithm of each side. a 


Let uy, = Card(X MA”) in such a way that fx(z) = Ensounz”. Using For- 
mula (8.17), we obtain for any n > 1 the formula known as Newton’s formula in 
the context of symmetric functions 


Sy = NUy + y SiUn—j- (8.18) 
1<i<n-1 
Since from Equation (8.17) we have fs(z) = peo. , we deduce that f(z) =zfy(z)+ 


f(z) fx (z), whence Formula (8.18). 

Let now P be the set of primitive necklaces in S and let p, = Card(PA”). Then 
since a word of S of length n is a power of a primitive word of length d with d dividing 
n and that this word has d conjugates, we have the following equality, generalizing 
Equation (8.11) 


Sn =) Pa- (8.19) 
d|n 


Like Equation (8.11), Equation (8.19) can be written as an equation relating power 
series and giving a generalization of the Cyclotomic Identity (8.12), namely, 


fet) =T1-— 


a (8.20) 
a ¢l —zi)Pn 

Let c, be the total number of necklaces in S, primitive or not. A word of length n 
in S is in a unique way a power of a primitive word of S. Thus cn = iain Pa We give 
below two examples of computation of sy), Pn, Cn. 
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Example 8.3.11 Let S be the set of representatives of necklaces on A = {a,b} with- 
out consecutive occurrences of b. Then S is the set of words having a conjugate in 
X* where X is the circular code X = {a,ba}. Thus, by Theorem 8.3.10, we have 

7 1 

~ laz- 22° 


Cs(z) 


By Newton’s formula, since u; = uz = 1 and uy, = 0 for n > 3, we have Sn41 = 
Sp t+5y—1 forn > 2. 
We obtain the values indicated in Table 8.8. 


n{/l 23 4 5 6 7 8 9 10 11 #=12~ «13 

sy | 1 3 4 7 11 18 29 47 76 123 199 322 521 

Poll 111 2 2 4 5 8 &I1 18 25 40 

om |122 3 3 5 5 8 10 15 19 31 41 
Table 8.8 


The values of sy, Pn,Cn forn < 13. 


The three necklaces of length 5 without bb (in agreement with cs = 3) are repre- 
sented in Figure 8.4. 


@ @ @ 
@ Q @ b) ©) () 


Qi @ QQ @ Q_ @ 


Figure 8.4 
The three necklaces of length 5 on the alphabet {a,b} without bb. 


Example 8.3.12 Let next S be the set of representatives of necklaces on A = {a,b} 
without occurrence of bbb. Then S is the set of words having a conjugate in X* where 


X is the circular code X = {a,ba,bba}. Thus 
= 1 
~ l-z-z 


Cs(z) 


and Spy. = Sy +Sp—-1+Sp—2 for n > 3. We obtain the values shown in Table 8.9. 
The four primitive necklaces of length 5 without bbb (in agreement with ps = 4) are 
represented in Figure 8.5. 


2 3 


oar < 
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n{/1 23 4 5 6 7 8 9 10 11 12 13 

s, | 1 3 7 11 21 39 71 131 241 443 2757 

Poi|l 1 2 2 4 5 10 15 26 42 74 121 212 

om |12 3 4 5 9 11 19 29 48 75 132 213 
Table 8.9 


The values of sy, Pn,Cn for the set of necklaces without bbb. 


Figure 8.5 
The four primitive necklaces of length 5 on the alphabet {a,b} without bbb. 


The formulae of this section generalize those of the previous one. MacMahon’s Iden- 
tity (8.13) also generalizes to 


1 
Cn = ~Y" p(n/d)sa 
Main 


where ¢ denotes Euler totient function. This allows a direct computation of the c,. 


8.4 Lyndon words 


A Lyndon word is a primitive word that is less than all its conjugates in the alpha- 
betic order. We denote by L the set of Lyndon words. 
The first Lyndon words on {a,b} are 
a,b 
ab 
aab,abb 
aaab, aabb, abbb 
aaaab, aaabb, aabab, aabbb, ababb, abbbb 


We first give the following equivalent definition. 


Proposition 8.4.1 A word is a Lyndon word if and only if it is strictly smaller than 
any of its proper suffixes. 
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Proof. — The condition is sufficient. Indeed, let w = uv with u,v nonempty. Since 
w <v, we have w < vu. 

It is also necessary. For w € L let w = uv with u,v nonempty. Assume first that v 
is a prefix of w and thus that w = vt. Since w is a Lyndon word, w < tv. But uv < tv 
implies u < tf and thus vu < vt, a contradiction. Thus v is not a prefix of w. But then 
v <w implies that vu < w, a contradiction. We conclude that w < v. a 


Note that, as a consequence, a Lyndon word is unbordered. Indeed, if u is both a 
nonempty suffix and prefix of w, then u < w and thus u = w by Proposition 8.4.1. 
The next statement gives a recursive way to build Lyndon words. 


Proposition 8.4.2 If 0,m © L with € < m, then ém is a Lyndon word. 


Proof. Let us first show that €m < m. If £ is a prefix of m, then m = ém’. Then 
m <a implies ém < €m' = m. Otherwise, € < m implies £m < m. 

Let v be a nonempty proper suffix of ém. If v is a suffix of m, then by Proposi- 
tion 8.4.1, m < v and thus £m < m < v. Otherwise, we have v = v'm. Then £ < v’ and 
thus &m < v'm = v. By Proposition 8.4.1, we conclude that ém € L. | 


For example, we have aab,ab € L with aab < ab and consequently aabab € L. 


8.4.1 The factorization theorem 


The following result is due to Lyndon (see [30] for more references). It motivated 
Knuth to call Lyndon words prime words in [26]. 


Theorem 8.4.3 Any word factorizes uniquely as a nonincreasing product of Lyndon 
words. 


The proof uses the following result. 


Lemma 8.4.4 Let ¢),...,€, be a nonincreasing sequence of Lyndon words and let 
w= ,---Ly. Then &; is the longest prefix of w, which is a Lyndon word, and £m, is 
the minimal nonempty suffix of w. 


Proof. Assume that ¢ € L is a prefix of w longer than ¢;. We have ¢ = ¢, --- €;u with 
i> 1 and wu anonempty prefix of @;,;. Then 0 <u < 0341 < €; < , a contradiction. 
Next, let v be the minimal suffix of w. Then v is in L by Proposition 8.4.1. There 
is an index j, anonempty suffix s of €; and a word t such that v = st. Then €, < 0) < 
§<st=v< Ly which implies v = yn. | 


Proof of Theorem 8.4.3. | We have to show that any word w can be written in a 
unique way w = ;---@,, with ¢,,...,2, € Land @; >... > n. 

Existence: Since the letters are in L, any word has a factorization in Lyndon 
words. Consider a factorization w = ¢; ---€, with m minimal. If @; < ¢;4 for some i, 
then w = 0 ---€-1(€6;41) +--+ 4m is a factorization in Lyndon words since ¢;;41 € L. 


502 Handbook of Enumerative Combinatorics 


Uniqueness: Assume that ¢)-+-¢, = @---@., with €;,@ EL, 0; >... > & and 
(,>...> 0 ,. By Lemma 8.4.4, we have ¢; = ¢,, which gives the conclusion by 
induction on m. | | 


We illustrate Theorem 8.4.3 by giving below the factorization of the word 
abracadabra. 


(abracad) (abr) (a) 


Let P be the set of prefixes of Lyndon words, also called preprime words in [26]. 

We call a word minimal if it is minimal for the lexicographic order in its con- 
jugacy class. Clearly, a word is minimal if and only if it is a power of a Lyndon 
word. 

A sesquipower of a word x is a word w = x”p with n > | and p a proper prefix 
of x. Set m = |w|. The word w is determined by x and m. It is called the m-extension 
of x. 

The following result appears in Duval [13]. 


Proposition 8.4.5 The set P is the set of sesquipowers of Lyndon words distinct of 
the maximal letter. 


The proof uses the following lemma. 


Lemma 8.4.6 For any word p and letter a such that pa is a prefix of a minimal word 
and for any letter b such that a < b, the word pb is in L. 


Proof. Let x be a Lyndon word such that pa is a prefix of x” for some n > 1. Then 
p=x"-1q and x = gar. 

We first show that if a < b, then gb € L. Indeed, this is true if g is empty. Oth- 
erwise, let tf be a proper suffix of g. Then tar is a proper suffix of x. By Proposi- 
tion 8.4.1, this implies x < tar and therefore g < t. Thus pb < tb. Since any proper 
suffix of pb is of this form, this shows that pb € L by Proposition 8.4.1 again. 


Now, since x < gb, we have x""qb € L for any m > | by Proposition 8.4.2. a 


Proof of Proposition 8.4.5. | Let x be a Lyndon word distinct of the maximal letter. 
Any sesquipower w of x is a prefix of a power x” of x. By hypothesis, we can write 
x = pagq with a not the maximal letter. Then, by Lemma 8.4.6, for any letter b > a, 
we have x"pb € L and thus w is in P. 

Conversely, we use an induction on the length of w € P. If |w| = 1, then w € L. 
Assume |w| > 1. Set w = va with a € A. By induction hypothesis, v = y"p with y € L, 
n> 1 and p proper prefix of y. Set y = pbu with b € A. Since w is a prefix of a Lyndon 
word, we have pb < pa and thus b < a. If a= b, then w is is a sesquipower of y. 

Finally if b <a, wis a Lyndon word by Lemma 8.4.6. a 


Observe that the Lyndon word x such that w is a sesquipower of x is unique. 
Indeed, assume that w is a sesquipower of x,x’ € L. Assuming that |x| < |x’|, we have 
x = x*p with p nonempty prefix of x. Then p <x <x’ < p, acontradiction. 
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8.4.2 Generating Lyndon words 


Proposition 8.4.5 can be used to generate Lyndon words of a given length in alpha- 
betic order (this algorithm is due to Fredericksen and Maiorana [17], and indepen- 
dently to Duval [14], see [26]). The idea is to generate all preprime words of this 
length. This generation problem has been considered in several contexts (see [37], 
[34], or [26] in particular). 

The algorithm SESQUIPOWERS is represented below. We use the alphabet 
{0,...,k —1}. This algorithm visits all preprime words a,---a, of length n with 
an index j such that aj +--+ a, is an extension of a) ---aj; (we say equivalently that the 
algorithm visits a}a2--+d, with index j or that the algorithm visits aja2---aj). 


SESQUIPOWERS(n,k) 
1 fori<+ 1tondo 


2; aj <— 0 

3 jl 

4 while true do 

5 > Visit a) -+-da, with index j 
6 jen 

7 while a; =k—1do 

8 jicj-l1 

9 if j = 0 then 

10 return 

11 aj<-ajt+1 

12 b> Nowa,:--a; EL 

13 fori «+ j+1tondo 

14 aj <— Gj 

15 > Make n-extension 


The assignment at line 11 makes aj ---a; a Lyndon word (by Lemma 8.4.6). The 
loop at lines 12-15 realizes the n extension of the word ay ---aj;. 

In particular, the sequence of words a ;a2---a; visited by the algorithm is the 
sequence of Lyndon words of length at most n in increasing order and the sequence 
of words ajd2--+d, visited with index n is the sequence of Lyndon words of length 
n in increasing order. 

We illustrate this on an example. Consider the list in alphabetic order of the words 
in P of length 5 plus b> (we read the list from top to bottom and then from left to 
right). The letter in boldface is at index j. 


aaaaa aabab abbab 
aaaab aabba abbba 
aaaba aabbb abbbb 
aaabb ababa bbbbb 
aabaa ababb 
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The six Lyndon words of length 5 are those with the marked letter at the last 
position. 

A possible variant of this algorithm enumerates preprime words in decreasing 
order. 


SESQUIPOWERSBIS(n,k) 
1 fori+1tondo 
2 aj«—k—-1 
3 aAn+1 «+ -l 
4 jel 
5 while true do 
6 > Visit a,,...,d@, with index j 
7 if a; = 0 then 
8 return 
9 aj <— aj— 1 
10 forh<« j+1tondo 
11 an<—k—-1 
12 jal 
13 h<e2 
14 while a;,_ ; < ay, do 
15 > Now ay ---aj_; is the (h — 1)-extension of aj ---a; 
16 if An—j < Ah then 
17 Jch 
18 h-h+l1 


At line 8, the assignement realizes the inverse of the operation at line 11 of 
SESQUIPOWERS. The loop at lines 13-17 implements the computation of the index j 
such that a  ---d, 1s a sesquipower of a; ---a;. It is guaranteed to always end by the 
assignment of line 3. 

Recently, Kociumaka, Radoszewski and Rytter have presented a polynomial time 
algorithm to compute the kth Lyndon word [27]. 


8.5 Eulerian graphs and de Bruijn cycles 
A de Bruijn cycle of order n on k letters is a necklace of length k” such that every 
word of length 7 on k letters appears exactly once as a factor. For example 
aabb 
aaababbb 
aaaabaabbababbbb 
aaaaabaaabbaababaabbbababbabbbbb 


are de Bruijn cycles of order 2,3,4,5. 
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Figure 8.6 
The de Bruijn graph of order n = 3. 


The de Bruijn graph of order on an alphabet A is the following labeled graph. 
Ithas A”! as its set of vertices. Its edges are the pairs (u,v) such that u = aw, v = wb 
with a,b € A. Such an edge is labeled b. The de Bruijn graph of orders 3,4 on the 
alphabet {a,b} are represented in Figure 8.6 and Figure 8.7. 


Figure 8.7 
The de Bruin graph of order n = 4. 


A cycle in a graph is an Euler cycle if it uses each edge of the graph exactly 
once. A finite graph is Eulerian if it has an Euler cycle. 

It is easy to verify that the de Bruijn cycles of order n are the labels of Euler 
cycles in the de Bruijn graph of order n. The following result shows the existence of 
de Bruijn cycles of any order. 


Theorem 8.5.1 A strongly connected finite graph is Eulerian if and only if each 
vertex has an indegree equal to its outdegree. 
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Proof. The condition is necessary since an Euler cycle enters each vertex as many 
times as it comes out of it. 

Conversely, we use an induction on the number of edges of the graph G. If there 
are no edges, the property is true. Let C be a cycle with the maximal possible number 
of edges not using twice the same edge. Assume that C is not an Euler cycle. Then, 
since G is strongly connected, there is a vertex x which is on C and in a non-trivial 
strongly connected component H of G\C. Every vertex of H has an indegree equal 
to its outdegree. So, by induction hypothesis, H contains an Eulerian cycle D. The 
cycles C and D have a vertex in common and thus can be combined to form a cycle 
larger than C, a contradiction. a 


We denote by d™ (v) the indegree of v (which is the number of edges entering v) 
and by d~ (v) its outdegree (which is the number of edges coming out of v). 

A variant of an Euler cycle is that of Euler path. It is a path using all the edges 
exactly once. It is easy to deduce from Theorem 8.5.1 that a graph has an Euler path 
from x to y if and only if d* (x) —d7 (x) =d7(y) —d*(y) =1 and dt (z) =d™ (z) for 
all other vertices. 

The computation of an Euler cycle along the lines of the proof of Theorem 8.5.1 
is an interesting exercise in recursive programming. It is realized by the following 
function EULER. 


EULER(s,t) 
1 if there exists an edge e = (s,x) still unmarked then 
2 MARK(e) 
3 c + (e, EULER(x,t)) 
4 return (EULER(s,5),c) 
5 else return empty 


The proof of correctness of this algorithm uses the following steps. The function 
computes an Eulerian path from s (the source) to ¢ (the target). It uses marks on the 
edges of the graph, which are initially all unmarked. 

It chooses an edge e = (s,x) leaving s. 

If there is an Euler path from s to ¢ beginning with e, the solution is 


(e, Euler(x, p)). 


Else the solution is 
(Euler(s,s),e,Euler(x, p)). 


The following result is due to van Aarden-Ehrenfest and De Bruijn [1]. We are 
going to see a derivation of it using linear algebra. 


Theorem 8.5.2 The number of de Bruijn cycles of order n on an alphabet with k 


letters is ; 


N(n,k) =k" (kK!) (8.21) 
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n | 1 y) 3 4 = 5 
N(n,2) | 1 1 2 16 512 
N(n,3)| 2 24 13824 

N(n,4) | 6 331776 

N(n,5) | 24 


Table 8.10 
Some values of the number N(n,k) of de Bruijn cycles of order n on k letters. 


In particular, for k = 2, there are 22" '—" de Bruijn cycles of order n. Table 8.10 lists 
some values of the numbers N(n,k). 

The result for k = 2 was obtained as early as 1894 by Fly Sainte-Marie (see [4] 
for a historical survey). 

Observe that N(1,k) = (k—1)!. This is in agreement with the fact that de Bruijn 
cycles of order | are the circular permutations of the k letters. 


8.5.1 The BEST theorem 


The following result, known as the BEST theorem, is due to van Aarden-Ehrenfest 
and de Bruin [1], and also to Smith and Tutte [40]. For a graph G on a set V of 
vertices, denote 2(G) = [],cy(d*(v) — 1)!. A spanning tree of G oriented towards a 
vertex v is a set of edges T such that, for any vertex w, there is a unique path from w 
to v using the edges in T. 


Theorem 8.5.3 Let G be an Eulerian graph. Let v be a vertex of G and let t(G) be 
the number of spanning trees oriented towards v. The number of Euler cycles of G is 


1(G)x(G). 


Proof. Let E be the set of Euler cycles and let E, be the set of Euler paths from 
vertex v to itself. Since each Euler cycle passes d*(v) times through v, we have 
Card(E,) = d* (v) Card(E). 

Let % be the set of spanning trees of G oriented towards v. We define a map 
o, : E, > & as follows. Let P be an Euler path from v to v. We define T = @(P) as 
the set of edges of G used in P to leave a vertex w # v for the last time. Let us verify 
that T is a spanning tree oriented towards v. 

Indeed, for each w # v, there is a unique edge in T going out of w. Continuing in 
this way, we reach v in a finite number of steps. Thus there is a unique path from w 
tov. 

Conversely, starting from a spanning tree T oriented towards v, we build an Euler 
path P from v to vas follows. We first use any edge going out of v. Next, from a vertex 
w, we use any edge previously unused and distinct from the edge in T, as long as such 
edge exists. There results an Euler path P from v to v, which is such that p(P) = T. 
This shows that Card(g~!(T)) = d*(v)! Tw. (d* (w) — 1)!. Consequently 


Card(E) = Card(E,) /d*(v) =t(v)z(v). 
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We illustrate Theorem 8.5.3 on the example of the de Bruijn graph of order 3 
(Figure 8.6). 


Example 8.5.4 Figure 8.8 represents the two possible spanning trees oriented to- 
wards bb in the de Bruijn graph of order 3. Following the Eulerian path in the de 
Bruijn graph of order 3 (see Figure 8.6), using in turn each of these spanning trees, 
starting and ending at the root, we obtain the two possible de Bruijn words 


aaababbb, abaaabbb. 


Figure 8.8 
The two spanning trees of de Bruijn graph of order n = 3 oriented towards bb. 


8.5.2 The Matrix-tree theorem 


Let G be a multigraph on a set V of vertices. Let M be its adjacency matrix defined 
by M,,, = Card(£,,,) with E,,, the set of edges from v to w. Let D be the diagonal 
matrix defined by Dy, = Y,cy Mw and let L = D—M be the Laplacian matrix of 
G. Note that the sum of the elements of each row of L is 0. We denote by K,(G) 
the determinant of the matrix C, obtained by suppressing the row and the column of 
index v in the matrix L. 

The following result is due to Borchardt [8]. 


Theorem 8.5.5 (Matrix-Tree Theorem) For any v € V the number of spanning 
trees of G oriented towards v is K,(G) 


Proof. Denote by N,(G) the number of spanning trees oriented towards v. 

We use an induction on the number of edges of G. The result holds if there are 
no edges. Indeed, if there is no edge leading to v, then N,(G) = 0. On the other hand, 
since the sum of each row of C, is 0, we have K,(G) = 0. Thus N,(G) = K,(G). 

Consider now an edge e from w to v. Let G’ be the graph obtained by deleting 
this edge and G” the graph obtained by merging v and w. 

We have 

N,(G) = N,(G’) +.N,(G"). (8.22) 


Indeed, the first term of the right-hand side counts the number of spanning trees ori- 
ented towards v not containing the edge e and the second one the remaining spanning 
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trees. Similarly, we have 
K,(G) = K,(G’) + K,(G"). (8.23) 


Indeed, assume v, w to be the first and second indices. The Laplacian matrices of the 
graphs G and G” have the form 


vy a a+b+c+d | x+y 
= Mis 
oa lls y Pi zt+t U 


The Laplacian matrix L’ of G’ being the same as L with c+1,d—1 instead of c,d. 
Then 


K,(G) > 


and thus Formula (8.23) by the linearity of determinants. By induction hypothesis, 
we have K,(G’) = N,(G’) and K,(G”) = N,(G”) By (8.22) and (8.23) this shows that 
K,(G) =N,(G). | 


Example 8.5.6 For the graph G of Figure 8.6, we have (the matrix C is obtained 
from L by suppressing the first row and the first column of L) 


1 -l 0 0O 


a or 
pe. Beh al. ea lea 0 


0 -1 1 


One has det(C) = 2 in agreement with Theorem 8.5.5 since, by Example 8.8, the 
graph G has two spanning trees oriented towards bb. 


It is possible to deduce the explicit formula for the number of de Bruijn cycles of 
Theorem 8.5.2 from the matrix-tree theorem. 

We denote by G* the edge graph of a graph G. Its set of vertices is the set E of 
edges of G and its set of edges is the set of pairs (e, f) € E x E such that the end of 
e is the origin of f. It is easy to verify that the edge graph of the de Bruijn graph G,, 
can be identified with G,,+1. 

A graph is regular of degree k if any vertex has k incoming edges and k outgoing 
edges. If G is regular, the number t(G) of spanning trees oriented towards a vertex v 
does not depend on v. 

The following result is due to Knuth [24] (see also [25], Exercise 2.3.4.2). 


Theorem 8.5.7 Let G be a regular graph of degree k with m vertices. Then 
t(G*) =K"—-)-h(G), 
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The proof uses the matrix-tree theorem. 
It is easy to prove Formula (8.21) by induction on n using this result (and the 
preceding ones). Indeed, by Theorem 8.5.3, and since G,, has k"—! vertices, we have 


N(n,k) = (k— 1)!" 't(G,). 
Thus (8.21) is equivalent to 
t(Ga) =k" (8.24) 
Assuming (8.24) and using Theorem 8.5.7, we have 


n—-l(p_1)_ 
Oni) = ke (Gy) 
cian al alan 


| a 1 Ke 
which proves that (8.24) holds for n+ 1. 


8.5.3. Lyndon words and de Bruijn cycles 
The following beautiful result is due to Fredericksen and Maiorana [17]. 


Theorem 8.5.8 Let 0) < l2 <...< £m be the increasing sequence of Lyndon words 
of length dividing n. The word €£2-+-€m is a de Bruijn cycle of order n. 


The original statement contains the additional claim that the de Bruijn cycle obtained 
in this way is lexicographically minimal. We shall obtain this as a consequence of a 
variant of Theorem 8.5.8 (see Theorem 8.5.10 below). 

For example, ifn = 4 and A = {a,b}, then 


aaaabaabbababbbb = a aaab aabb ab abbb b 


is a de Bruijn cycle of order 4. 
We will use the following lemma. 


Lemma 8.5.9 Let w be a prefix of length n of a Lyndon word and let ¢ be its longest 
prefix in L. Then w is the n-extension of &. 


Proof. 

Set w = és and let v be such that wv € L. Set also r = |¢|,n =|w| and wy = aq --- dn 
with a; € A. By Proposition 8.4.1, we have wv < sv. Thus there is some index ¢ with 
1 <t < |sv| such that aj = aj,, for 1 < j <t—1 and a < a4,. If t <n—r, by 
Lemma 8.4.6, the word a, ---a;+, is a prefix of w, which is a Lyndon word longer 
than ¢. Thus a; = aj;4, for 1 < j <n—r. This implies that 7 is a period of w and thus 
the conclusion. | 


Proof of Theorem 8.5.8. Since €1£2---€m has length k”, we only need to prove that 
any word w = a ---d, of length n appears as a factor of 0; ---£,0,£2. We denote by 
a the first letter of the alphabet and by z the largest one. We consider the following 
cases. 
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(a) Assume first that w is primitive and that w = uv with vu = ¢; and that u is not 
a power of z. Set u = pbq with p € z* and b a letter b < z. By Lemma 8.4.6, 
vpz € L. By repeated use of Lemma 8.4.2, vzl“l is a Lyndon word. Thus ¢;4.) < 
vzl“l. This implies that v is a prefix of €,,) and thus w is a factor of €;0;41. 


(b) Assume next that w = wv is primitive, that u € z* and that vu € L. We can first 
rule out the case where v € a*. Indeed, z/a”~/ is a factor of £n—1£n€100. Let k 
be the least index such that v < & (the existence of k follows from the fact that 
vu = €; for some j). Then  < vu and thus v is a prefix of ,. Let v’ < v be the 
Lyndon word such that v is a sesquipower of v’. 


(b1) Assume first that v’ A &_1. Let v” be the word v’ with its last 
letter changed into a. The word visited before v’ by Algorithm 
SESQUIPOWER(n,k) is, in view of Algorithm SESQUIPOWERBIS(n,k), 
the word v’z"-!"l, Thus &—1 ends with u, j begins with v and thus 
w = uv is a factor of C,_1 0x. 

(b2) Otherwise, v’ = ¢,_1. For the same reason as above, u is a suffix of 0,_. 
Since v is a sesquipower of v’, it is a prefix of v’v and thus also a prefix 
of £,_12,. Thus w is a factor of 0,20, 12. 


(c) Assume finally that w = (uv)¢ with d dividing n and vu = &,. 
(cl) Ifu ¢ z* then C4) < (vu)4—! yzlel since the latter is a Lyndon word. Thus 
w is a factor of 0,244. 
(c2) Otherwise, ¢,_; ends with at least (d — 1)|w| letters z and (4) < 
(vu)4—! lol, Thus w is a factor of €,_ 1 l;441. 


We illustrate the cases in the proof for n = 6 and A = {a,b}. Table 8.11 gives the 
sequence ¢,. 


k | 1 2 3 4 5 6 7 8 
f; | a aaaaab aaaabb aaabab aaabbb aab aababb  aabbab 


9 10 11 12 13 14 
aabbbb ab ababbb abb abbbbb b 


Table 8.11 
The Lyndon words of length dividing 6. 


(a) Let w = aabaaa. Then u = aab, v = aaa and vu = £2. We find w as a factor of 
L703. 
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(b1) Let w = baaaab. Then u = b and v = aaaab. We find k = 3, v =v andwisa 
factor of 053. 


(b2) Let w = bbabab. Then u = bb, v = abab. We find k = 11. We have v’ = ab and 
we find w as a factor of C9019 £11. 


(cl) Let w = (aba)*. Then u =a, v = ba and k = 6. We find w as a factor of 0607. 


(c2) Let w = (bab)?. Then u = b, v = ab and k = 12. We find w as a factor of 
11412613. 


Let X be a set of words. A de Bruijn cycle of order n relative to X is a necklace such 
that every word of X of length n appears exactly once as a factor. The usual notion 
of de Bruijn cycle is relative to X = A*. 

Consider for example the set X of words on {a,b} that are representatives of 
necklaces without consecutive occurrences of b (see Example 8.3.11). Then aaab is 
a de Bruijn cycle of order 3 relative to X and aaaabab of order 4. 

The following result, due to Moreno [34], gives a family of sets X for which 
there are de Bruijn cycles of any order relative to X. Let 0) < £2 <... < £m be the 
increasing sequence of Lyndon words of length dividing n. For s < m, we denote by 
X, the set of words such that no factor has a conjugate in {¢1,...,@s}. 


Theorem 8.5.10 For any s < m, the sequence ;l541-+++€m is a de Bruijn cycle of 
order n relative to X;. 


One can deduce from this result the fact that the de Bruijn cycle given by Theo- 
rem 8.5.8 is the minimal one for the alphabetic order (see [35]). 

As another variant of Theorem 8.5.8, let us quote the following result due to Yu 
Hin Au [2]: Concatenating the Lyndon words of length n in increasing order, one 
obtains a word that contains cyclically all primitive words of length n exactly once. 
For example, for n = 4 and A = {a,b}, one obtains the word aaab aabb abbb, which 
contains cyclically all 12 primitive words of length 4. 


8.6 Unavoidable sets 


A word t is said to avoid a word p if p is not a factor of f, i.e. if the pattern p does 
not appear in the text ¢. For example the word abracadabra avoids baba. The set 
of all words avoiding a given set X of words has been of interest in several contexts 
including the notion of a system of finite type in symbolic dynamics (see [29] for 
example). This notion has been extended to many other situations (see in particular 
the case of partial words in [7]). 

Let A be a finite alphabet. An unavoidable set on A is a set  C A* of words on the 
alphabet A such that any two-sided infinite word (a,),¢z on the alphabet A admits at 
least one factor in /. It is of course equivalent to ask that any one-sided infinite word 


Words 513 


has a factor in J or also, since the alphabet is finite, that the set of words that avoids 
Tis finite (see [31] for an exposition of the properties of unavoidable sets). 


Example 8.6.1 Let A = {a,b}. The set U = {a,b!°} is unavoidable since any word 
of length 10 either has a letter equal to a or is the word b'”. On the contrary, the set 
V = {aa,b!} is avoidable. Indeed, the infinite word (ab)® = ababababab ... has no 
factor inV. 


Proposition 8.6.2 On a finite alphabet, any unavoidable set contains a finite un- 
avoidable set. 


Proof. Indeed, let X be an unavoidable set and let S be the set of words avoiding 
X. Since X is unavoidable, S is finite. Let n be the maximal length of the words of S. 
Let Z be the set of words of X of length at most n+ 1. Every word of length n+ 1 
has a factor in X that is actually in Z. Thus Z is unavoidable. a 


The following gives an equivalent definition of unavoidable sets that holds for 
finite sets. It will be used below. 


Proposition 8.6.3 Let I C A* be a finite set of words. The following conditions are 
equivalent. 


(i) The set I is unavoidable. 


(ii) Each two-sided infinite periodic word has at least one factor in I. 


Proof. It is enough to show that (ii) > (i). Let (adn)nez be a two-sided infinite 
sequence of letters. Let u € A* be a word longer than any word in J and having an in- 
finite number of occurrences in the sequence (a,),cz. This sequence has at least one 
factor of the form uvu. By the hypothesis, the infinite periodic word ...uvuvuvuv... 
has a factor w € J. The word w is a factor of at least one of the words uv or vu. It is 
thus also a factor of the sequence (ay)n¢z and thus J is unavoidable. i 


This statement is false if J is infinite. For example, on a three-letter alphabet, the set 
of squares is avoidable but every periodic word contains obviously a square. 


8.6.1 Algorithms 


To check in practice that a given finite set X is unavoidable, there are two possible 
algorithms. 

The first one consists in computing a graph G = (P,E), where P is the set of 
prefixes of X and E£ is the set of pairs (p,s) for which there is a letter a € A such that 
s is the longest suffix of pa which is in P. 


Proposition 8.6.4 A finite set X is unavoidable if and only if every cycle in G con- 
tains a vertex in X. 


514 Handbook of Enumerative Combinatorics 


Proof. For each integer n > 0, and vertices u,v € P, there is a path of length n from 
u to v if and only if there exists a word y of length n such that v is the longest suffix 
of uy in P. This can be proved by induction on n. It follows that there is a path of 
length n from € to a vertex in X if and only if A*X NA” £0. a 


Figure 8.9 
The graph for X = {a,bb}. 


Example 8.6.5 For X = {a,bb}, the word graph is given in Figure 8.9. By inspec- 
tion, the set X is unavoidable. 


The second algorithm is sometimes easier to write down by hand. Say that a set 
Y of words is obtained from a finite set of words X by an elementary derivation if 


(i) either there exist words u,v € X such that u is a proper factor of v, and 


Y=X\v 


(ii) or there exists a word x = ya € X with a € A such that, for each letter b € A 
there is a suffix z of y such that zb € X, and 


Y = (X Uy) \x. 


A derivation is a sequence of elementary derivations. We say that Y is derived 
from X if Y is obtained from X by a derivation. 


Example 8.6.6 Let X = {aaa,b}. Then we have the derivations 


X — {aa,b} > {a,b} > {€,b} — {e} 


where the first three arrows follow case (ii) and the last one case (i). 
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The following result shows in particular that if Y is derived from X, then X is 
unavoidable if and only if Y is unavoidable. We denote by Sy the set of two-sided 
infinite words avoiding X. 


Proposition 8.6.7 If Y is derived from X, then Sx = Sy. 


Proof. Itis enough to consider the case of an elementary derivation. In the first case 
where Y = X \v, where v has a factor in X, then clearly Sy = Sy. In the second case, 
we clearly have Sy C Sy since Y is obtained by replacing an element of X by one of 
its factors. Conversely, assume by contradiction the existence of some s € Sy \ Sy. 
The only possible factor of s in Y is y. Let b be the letter following y in s. Then s has 
a factor in X, namely zb where z is the suffix of y such that zb € X whose existence 
is granted by the definition of a derivation. This is a contradiction. | 


The notion of a derivation gives a practical method to check whether a set is 
unavoidable. We have indeed the following result. 


Proposition 8.6.8 A finite set X is unavoidable if and only if there is a derivation 
from X to the set {€}. 


Proof. Let X # {€} be unavoidable. We prove the existence of a derivation to {€} 
by induction on the sum /(X) of the lengths of words in X. If € € X, we may derive 
{e} from X. Thus assume € ¢ X, and let w be a word of maximal length avoiding 
X. For each b € A there is a word x, = zb € X that is a suffix of wh. Let xg = ya be 
the longest of the words x,. Then the hypotheses of case (ii) are satisfied and thus 
there is a derivation from X to a set Y with /(Y) < /(X). The converse is clear by 
Proposition 8.6.7. a 


In practice, there is a shortcut that is useful to perform derivations. It is described 
in the following transformation from X to Y. 


(iii) There is a word y such that ya € X for each a € A and 
¥ =(XUy)\ {ya [a € A}. 


It is clear that such a set Y can be derived from X and thus, we do not change the def- 
inition of derivations by adding case (iii) to the definition of elementary derivations. 
We use this new definition in the following example. 


Example 8.6.9 Let X = {aaa,aba,abb,bbb}. We have the following sequence of 
derivations (with the symbol a in the word x = ya underlined at each step) 


{aaa,aba,abb,bbb} -—> {aaa,ab,bbb} 
+ {aa,ab,bbb} —+ {a,bbb} + {a,b} 
+> {a,b} > {e} 


Derivations could of course be performed on the left rather than on the right. 
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8.6.2 Unavoidable sets of constant length 


In the sequel, we will be interested in unavoidable sets made of words having all the 
same length n. The following proposition is easy to prove. 


Proposition 8.6.10 Let A be a finite alphabet and let I be an unavoidable set of 
words of length n on A. The cardinality of I is at least equal to the number of conju- 
gacy classes of words of length n on the alphabet A. 


Proof. Let u € A* be a word of length n. The factors of length n of the word u® are 
the elements of the conjugacy class of u. Thus J must contain at least one element of 
this class. a 


We are going to prove the following result, which shows that the lower bound 
c(n,k) on the size of unavoidable sets of words of length n on k symbols is reached 
for alln,k > 1. 


Theorem 8.6.11 For all n,k > 1, there exists an unavoidable set formed of c(n,k) 
words of length n on k symbols. 


This result has been obtained by J. Mykkelveit [36], solving a conjecture of 
Golomb. His proof uses exponential sums (see below). Later, and independently, it 
was solved by Champarnaud, Hansel and Perrin [11] using Lyndon words. We shall 
present this proof here. 

It may be convenient for the reader to reformulate the statement in terms of 
graphs. A feedback vertex set in a directed graph G is a set F of vertices containing 
at least one vertex from every cycle in G. Consider, for n > 1, the de Bruijn graph 
G,+1 of order n+ 1 on the alphabet A whose vertices are the words of length n on A 
and the edges are the pairs (au,ub) for alla,b € A and u€ A” !. It is easy to see that 
a set of words of length n is unavoidable if the corresponding set of vertices is a feed- 
back vertex set of the graph G,.1. Thus, the problem of determining an unavoidable 
set of words of length k of minimal size is the same as determining the minimal size 
of a feedback vertex set in G,.1. The problem is, for general directed graphs, known 
to be NP-complete (see [18] for example). 

As a preparation to a proof of Theorem 8.6.11, we introduce the following no- 
tions. 

A division of a word w is a pair (¢',u) such that w = ¢‘u where £ € L, i> 1 and 
u € A* with |u| < |é|. 

By Proposition 8.4.5 each word in P admits at least one division. We say that a 
Lyndon word ¢ € L meets the word w if there is a division of w of the form (¢’,w). It 
is clear that for any @ € L there is at most one such division of w. 

The main division of w € P is the division (¢',w) where @ is the shortest Lyndon 
word that meets w. The word ¢' is the principal part of w, denoted by p(w), and u 
is the remainder, denoted by r(w). 

For example, with a < b, the word aabaabbba admits two divisions, which are 
(aabaabbb,a) and (aabaabb, ba). The first one corresponds to its decomposition as 
a sesquipower of a Lyndon word. The second one is its main division. 
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M, Gi 
daaaaaa | aaaaaaa aababab | abaabab 
aaaaaab | aaaaaab aababbb | bbaabab 


aaaaabb | baaaaab aabbabb | abbaabb 
aaaabab | abaaaab aabbbab | babaabb 
aaaabbb | bbaaaab aabbbbb | bbbaabb 
aaabaab | aabaaab abababb | bababab 
aaababb | abbaaab ababbbb | bbababb 
aaabbab | babaaab abbabbb | babbabb 
aaabbbb | bbbaaab abbbbbb | bbbabbb 
aabaabb | baabaab bbbbbbb | bbbbbbb 


Table 8.12 
The sets M7 and J7. 


Let n > 1 be an integer and let M,, be the set of minimal words of length n. For 
each m € M,, let p(m) be its principal part and r(m) its remainder. Let J, be the set 


I, = {r(m)p(m)|m € M,}. 
We remark that any minimal word that is not primitive appears in Jk. 


Example 8.6.12 Table 8.12 lists the elements of M7 and I, with the remainder of 
each word of Mz in boldface. 


The object of what follows is to show that J, is an unavoidable set. By Proposi- 
tion 8.6.10, the number of elements of [,, is the minimal possible number of elements 
of an unavoidable set of words of length n. 

Theorem 8.6.11 will be obtained as a consequence of the following one, giving a 
construction of the minimal unavoidable sets. 


Theorem 8.6.13 Let A be a finite alphabet and letn > 1. Let M,, be the set of words 
on the alphabet A of length n and that are minimal in their conjugacy class. For every 
word m € M,, let p(m) be the principal part of m and let r(m) be its remainder. Then 
the set 

In = {r(m)p(m)|m © My} 


is an unavoidable set. 
To prove Theorem 8.6.13, we need some preliminary results. 


Proposition 8.6.14 Let ¢ and m be two Lyndon words, with £ a prefix of m. Let s € A* 
be a proper suffix of m, with |s| < |¢|. Then for alli > 0, the word w = ¢'s is a Lyndon 
word. 


Proof. Lett be a proper suffix of w. Three cases may arise. 
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1. One has |t| < |s|. Then ¢ is a proper suffix of the Lyndon word m and thus 
t>m> £and since |t| < |é|, we have t > £"s = w. 


2. One has |r| > |s| and the word r factorizes as t = ¢/s, with 0 < j <i. Since s 
is a proper suffix of m, we have s > m > ¢. Consequently t = @/s > €/*! and 
since |s| < |¢|, we have t > @'s = w. 


3. One has |t| > |s| and the word f factorizes as t = s’t’, where s’ is a proper suffix 
of £. Since £ € L, one has s’ > £, and consequently t = s’t/ > Us =w. 


In all cases t > w and thus w is a Lyndon word. a 


Proposition 8.6.15 Let w be a prefix of a minimal word and let (¢!,u) be its main 
division. Let u' € A* be a word of the same length as u such that the word w' = ¢iu' 
is also a prefix of a minimal word. Then the main division of w' is the pair (¢',u’). 


Proof. Let (m/,v) be the main division of w’. We have w! = m/v with |v| < |ml. 
Since (¢',u’) is a division of w’, the word m is a prefix of ¢. We are going to show by 
contradiction that m cannot be a proper prefix of @. 

Suppose that m is a proper prefix of @. Since the factorization of a minimal word 
as a power of a Lyndon word is unique, we cannot have the equality m/ = ¢'. Suppose 
first that |m/| < |£'|. Since w! = m/v = fu’, the word m/ is a proper prefix of the word 
¢’, Thus there exists a non-empty word x € A* such that m/x = ¢! and xu’ = v. We 
thus have 

w= lu =m’ xu. 
Since |xu| = |xu'| = |v| < |ml, the pair (m/,xw) is a division of w, which is a contra- 
diction since m is a proper prefix of £ and (¢',) is the main division of w. 

Let us now suppose that |m/| > |¢'|. Since w! = m/v = fu’, the word ¢ is a proper 
prefix of the word m/. Since m is a proper prefix of @, there exists an integer k > 0 
and a prefix m’ of m such that = m*m/’. Since ¢ is a primitive word, m’ is non- 
empty. As a consequence, £ admits m’ both as a non-empty prefix and suffix, which 
is contradictory since @ is a Lyndon word. | 


The final property needed to prove Theorem 8.6.13 is the following. 


Proposition 8.6.16 Let m be a Lyndon word and n a positive integer. Let N > 1 be 
the smallest integer such that |m%| >n. Then the word mN*! has a factor in In. 


Proof. Let w be the prefix of length n of m”. Let (¢',u) be the main division of 
w. If u is the empty word, then, by construction, w € J, and the proposition is true. 
Suppose that u is not empty. 

The word ¢ is a prefix of m since either |w| < |m| or w admits a division of the 
form (m/,m’). Let s be the suffix of m having the same length as u. By Proposi- 
tion 8.6.14, the word ¢‘s is a Lyndon word. Thus, by Proposition 8.6.15, the main 
division of ¢‘s is the pair (¢',s). Consequently, the word sé! belongs to J,. But this 
word is a factor of nV+!, Thus m+! has a factor in I,. | 
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We are now able to prove Theorem 8.6.13. By Proposition 8.6.3, it is enough to 
show that every periodic two-sided infinite word of the form ...uuuuu... has at least 
one factor in [,. We may suppose without loss of generality that u is a Lyndon word. 
Let N be the least integer such that N|u| > n. Then, by Proposition 8.6.16, the word 
ut! has a factor in J,,. Thus J, is unavoidable. 


8.6.3 Conclusion 


The proof of J. Mykkeltveit in [36] is based on the following principle, presented 
in the case of a binary alphabet. Let us associate to a word w = dgd) ---dayn—1 on the 
alphabet {0,1} the sum s(w) = Yaj@/ where @ = e”'/". We denote by Is(w) the 
imaginary part of s(w). It can be shown that for each conjugacy class of words, only 
two cases occur: 


(i) either all words w are such that Js(w) = 0 (and then, for n > 2 one has actually 
s(w) = 0 for each of them) 


(ii) or there is, in clockwise order, one block of words w such that Is(w) > 0 fol- 
lowed by one block of words w such that Is(w) < 0 separated by at most two 
words w such that [s(w) = 0. 


Consider the set S,, of words of length n formed of 


(i) a representative of each conjugacy class of words w of length n such that 
Is(w) = 0 for all the conjugates. 


(ii) the words w = aga -+-dy_1 of length n such that Is(w) > 0 for the first time 
clockwise. 


It is shown in [36] that this set is unavoidable for all n > 2. The compari- 
son with the previous family of minimal unavoidable set shows that the fami- 
lies have nothing in common. The sets obtained are indeed different. The sets 
defined by J. Mykkeltveit have a slight advantage in the sense that the maxi- 
mal length of words avoiding the set is less. For example, for n = 20, there 
are 256 words of length 2579 that avoid J,, but none of length 563 that avoid 
all of S, (and there is a unique way to avoid S, with length 562). This com- 
putation has been performed using D. Knuth’s program UNAVOIDABLE2 (see 
http: //www-cs-faculty. stanford. edu/~knuth/programs.htm1). Our proof 
has the advantage of using only elementary concepts and in particular no real or 
complex arithmetic. 

Another proof of Theorem 8.6.11 obtained by the first two authors of [11] and 
presented in [10] is a construction working by stages. To explain these stages, let us 
consider the case of a binary alphabet A = {a,b}. Given a set X of two-sided infinite 
words, we say that a set Y of words is unavoidable in X if every word of X has a 
factor in Y. 

For i > 1, let X; be the set of two-sided infinite words on A that avoid a’. Let 
ci(n,k) be the number of conjugacy classes of words x of length n on k symbols such 
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123 45 6 7 8 9 10 
1}1 1 1 1 1 Ft 1 1 1 1 
2}/1 223 3 5 5 8 10 15 
3/1 23 45 9 11 19 29 48 
4/1 2 3 5 6 11 15 27 43 59 
S5);1 2 3 5 7 12 17 31 S51 91 
6/1 2 3 5 7 13 18 33 55 99 
7}1 2 3 5 7 13 19 34 57° 103 
8}/1 2 3 5 7 13 19 35 58 105 
9}1 2 3 5 7 13 19 35 59 106 
10};1 2 3 5 7 13 19 35 59 107 
Table 8.13 
The values of c;(n,2). 
that the words of the form x$ =---xxx--- are in X;. It is thus also equal to the number 


of orbits of period n in X;. Table 8.13 gives the values of c;(n,2) for 1 <i < 10 and 
1 <n < 10. The rows are indexed by i and the columns by n. Thus the second row 
is the last row of Table 8.8 and the third row is the last row of Table 8.9. Moreover, 
subtracting 1 from the first eight entries of the last three rows, we obtain the second 
row of Table 8.7 (we have to subtract 1 because a” is missing). 

The idea of the step-by-step construction of a minimal unavoidable set of words 
of length n is to construct a sequence Yj C Yo C ... CY, of sets of words of length 
n such that for 1 <i <n, the set Y, is unavoidable in X; with c;(n,k) elements. This 
can be stated as the following result. 


Theorem 8.6.17 For eachk > 1, andn > i> 1, there exists a set of c;(n,k) words of 
length n on k symbols that is unavoidable in Xj. 


The alternative proof consists in showing directly that the c;(n,k) last elements of J, 
form a set unavoidable in X;. 

It is interesting to remark that not all minimal unavoidable sets are build in this 
way. Indeed, there are sets that are minimal unavoidable in X;,4; but do not contain a 
minimal unavoidable set in X,. 

For example, let ¥; = {bbb}, Y2 = {bbb, bab}, Y3 = {bbb, bab, aab}. Then each 
Y; for 1 <i <3 is unavoidable in X; of size c;(3,2) and 1, = Y3U {aaa}. In particular, 
the set /3 contains an unavoidable set in X2 with two elements, namely Y2. However, 
the set J3 = {aaa,aba,bba, bbb} obtained from J; by exchanging a and b does not 
contain a two element set unavoidable in X>. 

A set of the form X; is a particular case of what is called a system of finite type. 
This is, by definition, the set of all two-sided infinite words avoiding a given finite 
set of words (see [29]). We do not know in general in which systems of finite type it 
is true that for each n there exists an unavoidable set having no more elements than 
the number of orbits of period n. 
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8.7. The Burrows—Wheeler transform 


The Burrows—Wheeler transform is a popular method used for text compression [9]. 
It produces a permutation of the characters of an input word w in order to obtain a 
word easier to compress. The presentation given here is close to that of [12]. 

Suppose w is a primitive word over a totally ordered alphabet A. Let 
W1,W2,...Wn be the sequence of conjugates of w in increasing lexicographic order. 
Let M(w) be the matrix having w1,w2,...,.W, as rows. For example, if w = aabacacb, 
the matrix M(w) is 

5 ’ 
a a 


M(aabacacb) = 


aa rere 8 

gs 8. 89090 &A 
saan ana Faa & 
gara ©aaaasd 

song aga maa 
gs aovra Faaea 
qarsg gagagoa 
sag gqno0 & 


b b 


The Burrows-Wheeler transform T(w) of w is the last column of M(w), read 
from top to bottom. If b; denotes the last letter of the word w;, for i= 1,2,...,n, then 
T(w) = bb2...bp. For instance, T (aabacacb) = babccaaa. 

It is clear that T(w) depends only on the conjugacy class of w, i.e. T(w) = T(w’) 
if w and w’ are conjugate. Therefore we may suppose that w is a Lyndon word, i.e. 
w=wy. 

The matrix M(w) defines a permutation o,, (or simply o when no confusion 
arises) of 1,2,...,n: 


O(i) = j > wj =4j---anay-+-aj-1. (8.25) 


In other terms, o(i) is the rank in the lexicographic order of the ith circular shift of 
the word w. For instance, for w = aabacacb, we have: 


Pre gi eee ne ae 
ch, De Ge B57 Oe Be a 


Let F(w) denote the first column of the matrix M(w). If c; denotes the first let- 
ter of the word w;, for i = 1,2,...,n, then F(w) = c,c2--+Ccy is the nondecreasing 
rearrangement of w. By definition, we have, for each index i, with 1 <i<n, 


qj = Co(i): (8.26) 


The permutation o transforms the first column of M(w) into its first row, i.e. into 
the word w. We have also the following formula expressing T(w) using o: 


Dp its Figs (8.27) 
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Indeed, b,j) is the last letter of w6,j) 
which is equivalent to the above formula. 

Given a primitive word w € A*, let 2,,, or simply z when no confusion arises, be 
the permutation defined by 


= 4j+++G,Q,+++a;_-1, hence boii) = aj-1, 


n(i) = o(o !(i) +1), (8.28) 
where the addition is to be taken modulo n. 


Remark 8.7.1 Observe that 7 is just the permutation defined by writing o as a word 
and interpreting it as a n-cycle. Thus we have also o(i) = 2’~!(1) and 


qj = Cri-1(1)+ 
In the previous example we have, written as a cycle, 
m= (12637485) 


and as an array 


_f 123 45 67 8 
PEND GS) Beale 28, A: G21) 
The following proposition is fundamental for defining the inverse transform. 


Proposition 8.7.2 If cjco-+-cy and bi b...by are the first and the last columns, re- 
spectively, of the matrix M(w), then cj = by), fori=1,2,...,n. 


Proof. Substituting in Formula (8.27) the value a; given by Formula (8.26), we 
obtain bj = Cg(g-1(;)—1), Which is equivalent to the statement of the proposition. ff 


The previous proposition states that the permutation 7,, transforms the last col- 
umn of the matrix M(w) into the first one. Actually, it can be noted that z,, transforms 
any column of the matrix M(w) into the following one. 


8.7.1 The inverse transform 


We now show how the word w can be recovered from T(w). For this we prove a 
property of the matrix M(w) stating that, for any letter a € A, its occurrences in F (w) 
appear in the same order as in T(w), i.e. the kth instance of a in T(w) corresponds 
(through 7) to its kth instance in F(w). In order to formalize this property we intro- 
duce the following notation. 

The rank of the index i in the word z = z1Z2--+Z,, denoted by rank(i,z), is the 
number of occurrences of the letter z; in z1Z2 --- z;. For instance, if z = babccaaa, then 
rank(4,z) = 1 and rank(6,z) = 2. 


Proposition 8.7.3 Given the words T(w) = biby---by and F(w) = cic2-++Cp, for 
each index i = 1,2,...,n, we have 


rank (i, F (w)) = rank(a(i),T(w)). 


Words 523 


Proof. | We first note that, for two words u,v of the same length, and for any letter 
a € A, one has 
au < av => ua < va. 


Thus, for all indices i, j, i<j and cj =c; implies 2(i) < 2(j). Hence, the number of 
occurrences of cj in c1c2---c; is equal to the number of occurrences of br(i) =c; in 
bi by -+-byyj). 


To obtain w from T(w) = b)b2...bn, we first compute F(w) = c1C2...cn by rear- 
ranging the letters b; in nondecreasing order. Proposition 8.7.3 shows that 7(i) is the 
index j such that c; = b; and rank(j,T (w)) = rank(i, F(w)). This defines the permu- 
tation 7, from which o can be obtained expressing 7 as a n-cycle, and then, by using 
Formula (8.26), the word w can be reconstructed. 


Remark 8.7.4 Proposition 8.7.3 further shows that the permutation 7 is related to 
the standard permutation of the word T(w). Recall that the standard permutation 
of a word v = bi b2---by on a totally ordered alphabet A is the permutation Tt such 
that, for i,j € 1,2,...,n, the condition t(i) < t(j) is equivalent to bj < bj or bj =b; 
and i < j. The permutation t may be obtained by numbering from left to right the 
letters of v, starting from the smallest letter, then the second smallest, and so on. For 
example, for v = babccaaa, we have that, written as a word, T = 51678234, and as 


an array: 
px: 123 45 67 8 
~“\5 167 8 2 3 4 ')° 
It is easy to see that the permutation z corresponds to the inverse of the standard 
permutation of T(w). 


Remark 8.7.5 The Burrows—Wheeler transform T(w) of a primitive word w de- 
pends only on the conjugacy class of w. Therefore, T defines an injective mapping 
from the primitive necklaces over an alphabet A to the words of A*. However such 
a mapping is not surjective. Remark 8.7.4 indeed shows that, if we consider a word 
u € A* such that the standard permutation t of u (and then also the permutation 
m= = 7!) is not a cycle, then there does not exist any word w such the T(w) = wu. Let 
us, for instance, consider the word u = bccaaab. Its standard permutation 


es 123 45 67 
“\4 67 12 3 °5 
is the product of two cycles 


t =(14)(26375). 


It follows that there does not exist any word w such that T(w) = u. 
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8.7.2 Descents of a permutation 


A descent of a permutation 7 is an index i such that z(i) > m(i+ 1). We denote 
by Des(z) the set of descents of the permutation 2. Consider the permutation 7, 
corresponding to a word w. It is clear from Proposition 8.7.3 that if i is a descent 
of 2, then c; 4 cj,1. Thus the number of descents of 7, is at most equal to k — 1, 
where k is the number of distinct symbols appearing in the word w. For instance, 
for w = aabacacb, %,(4) > %,(5), moreover 4 is the only descent of 7, and so 
Des(ty) = {4}. 

Let A = {a1,a2,...,a,} be a totally ordered alphabet with aj < a2 <... < 
ay. If w is a word of A*, denote by P(w) the Parikh vector of w: P(w) = 
(|Wla,>|Wla>--+>[Wla,)- It is clear that, if two words are conjugate, then they have 
the same Parikh vector, and so one can define the Parikh vector of a necklace. We 
say that a vector V = (1,N9,...,N,) is positive if n; > 0 for i= 1,2,...,k. We denote 
by p(V) the set of integers p(V) = {m,n +n2,...,21 +...+mx_1}. When V is pos- 
itive, p(V) has k— 1 elements. Let 7,, be the permutation corresponding to word w 
and let P(w) be the Parikh vector of w. It is clear from Proposition 8.7.3 that we have 
the inclusion Des(m,) C p(P(w)). 


Example 8.7.6 The Parikh vector of the word w = aabacacb is V = (4,2,2) and 
p(V) = {4,6}. The permutation 1, corresponding to w is 


af LES AS GOT SB 
GB A Be eae Sy 
Thus Des(ty) = {4} C p(V). 


The following statement, due to Crochemore, Désarménien and Perrin [12], re- 
sults from the previous considerations and Remark 8.7.4. 


Theorem 8.7.7 For any positive vector V = (nj,no,...,N), withn =n, +n2+...+ 
nk, the map w — Ty is one-to-one from the set of primitive necklaces of length n with 
Parikh vector V onto the set of cyclic permutations % on {1,2,...,n} such that p(V) 
contains Des(m). 


This result actually is a particular case of a result stated in [31] and closely re- 
lated to the Gessel-Reutenauer bijection introduced in the next section. Since each 
conjugacy class of primitive words can be represented by a Lyndon word, Theorem 
8.7.7 establishes a bijection between Lyndon words and cyclic permutations having 
special descent sets. The extension of this result (Theorem 11.6.1 of [31]) establishes 
a bijection between words and permutations, relating the Lyndon factorization of 
words and the cycle structure of the permutations. 
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8.8 The Gessel-Reutenauer bijection 


We have shown that the Burrows—Wheeler transform T(w) of a word w depends 
only on the conjugacy class of w. Therefore, T defines an injective mapping from 
the primitive necklaces over an alphabet A to the words of A*. However (cf. Remark 
8.7.5) such a mapping is not surjective. 

We now extend the Burrows—Wheeler transform by defining a bijective mapping 
® from the multisets of primitive necklaces over a totally ordered alphabet A to the 
words of A*. This bijection has been introduced by Gessel and Reutenauer in [20]. 
In order to define the mapping ® we introduce a new order on A*. For a word z € A*, 
z® denotes the infinite word zzz--- obtained by infinitely iterating z. Given u,v € A*, 
u <q v if and only if u® < v® in the lexicographic order. Remark that the order <q 
is different from the usual lexicographic order: for instance, aba <q ab. 

Following [33] (see also [23]), we give here a presentation of the mapping ® that 
emphasizes its relation with the Burrows—Wheeler transform. 

Let S = {51,52,...,5m} be a multiset of necklaces, represented by their Lyndon 
words, i.e. sz is the Lyndon word corresponding to the kth necklace of S. In some 
cases, it is convenient to denote by (z1z2---z) the necklace containing the word 
z1Z2°++Z. Denote also by n = |s;|+|s2|+...+|5m| the size of S and by L the least 
common multiple of the lengths of the words in S. 

In order to sort the elements in the necklaces of S according to the order <q, 
consider the collection of all words of the form ue \\el where u is an element of a 
necklace. All these words then have a common length L. We order this set of words 
lexicographically to yield a matrix M(S) with n rows and L columns. 


Example 8.8.1 Let S = {aab,ab,abb}. Then n = 8 and L = 6 and 


aabaaé b 
abaaoba 
abaobaé b 
abbaeb b 
M\S) = baabaqa 
babaoba 
babbaé b 
b ba bbea 


The transform ®(S) = b,b2---b, corresponds to the last column of the matrix 
M(S), read from top to bottom. In the previous example, ®(S) = babbaaba. 

It is clear that if the multiset S has only one necklace represented by its Lyndon 
word w, then ®(S) = T(w), ie. ®(S) corresponds to the Burrows—Wheeler transform 
of w. Note also that, if there are non trivial multiplicities in the multiset S, then there 
are repeated rows in the matrix M(S). 

Several properties of the Burrows—Wheeler matrix M(w) of a word w can be 
easily extended to the matrix M(S). In particular, if we denote by F(S) = cc2°++Cp 
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the first column of the matrix M(S), by using the same argument as in the proof of 
Proposition 8.7.3, it can be shown that, for any letter a € A, its occurrences in F(S) 
appear in the same order as in ®(S). This defines a permutation 7 that transforms the 
last column of the matrix M(S) into the first one. Actually (cf. Remark 8.7.4), 7 is 
the inverse of the standard permutation of the word ®(S). 


Example 8.8.2 The inverse of the standard permutation of the word babbaaba is 


of 1, 234 S56 TS 
SN Oe SG og TB a, 


In order to reverse the Burrows—Wheeler transform, given the word T(w) = 
b,bz--+by, we considered in Section 8.7.1 the inverse z of its standard permutation, 
then we expressed it as an-cycle (j1, j2,.-.-, jn) and we associated to this n-cycle the 
necklace (cj, Cj,°°*¢j,)- 

Recall that the Burrows—Wheeler transform is injective, but not surjective. This 
is a consequence of the fact that, for some words u, the inverse of its standard per- 
mutation cannot be expressed by a single n-cycle, but its decomposition contains 
several cycles (cf. Remark 8.7.5). This remark is at the base of the surjectivity of the 
Gessel-Reutenauer transform. 

Now we show how to reverse the transform ®, that is how the multiset of neck- 
laces S can be recovered from the word ®(S) = b,b2---bn. 

As for the Burrows—Wheeler transform, first compute the first column F(S) = 
€1C2**+Cy of M(S) by rearranging the letters b; in nondecreasing order. 

Then, consider the inverse 2 of the standard permutation associated to the 
word ®(S) = bib2---b,. With each cycle (j1, j2,..., j;) of 2, associate the necklace 
(cj, Cj. °*+¢;,). The multiset S is given by 


S = {(cjej-+-¢4) (dts das-+-+i) is a cycle of x}. 


Remark that different cycles of 2 could give rise to the same necklace, and this 
explains the use of multisets. 


Example 8.8.3 Let ®(S) = babbaaba (see Example 8.8.1). Rearranging the letters 
in nondecreasing order, one obtains F (S) = aaaabbbb. Then the permutation 7 is 


fe 2 BHk S 6-7 
MN toe eee rs SO Ga. SS 


By decomposing 7 in cycles 
m = (125)(3 6)(48 7), 
one obtains the multiset of necklaces S = {(aab), (ab), (abb)}. 


The following theorem, due to Gessel and Reutenauer [20], results from the pre- 
ceding considerations. 


Theorem 8.8.4 The map ® defines a bijection between words over a totally ordered 
alphabet A and multisets of primitive necklaces over A. 


A similar, but different, bijection has been proved in [19], where, instead of the 
lexicographic order, is used the alternate lexicographic order. 
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8.8.1 Gessel-Reutenauer bijection and de Bruijn cycles 


In this section we present an interesting connection, pointed out in [23], between the 
Gessel-Reutenauer bijection and the de Bruijn cycles. 

A multiset S = {51,52,...,5m} of necklaces is a de Bruijn set of span n over an 
alphabet A if |51| + |s2| +... + |Sm| = Card(A)” and every word w € A” is a prefix of 
some power of some word in a necklace of S. 


Remark 8.8.5 The number of distinct prefixes of length n of powers of the words in 
the necklaces of S is at most Card(A)”. So, given that S is a de Bruijn set of span n, 
every word in A” can be read exactly once within the necklaces of S. It also follows, 
in particular, that no two necklaces in S are equal, so that S is indeed a set, as opposed 
to a multiset, of necklaces. 


Remark 8.8.6 If Sis ade Bruijn set of span n, then S contains a necklace of length at 
least n. To show this, consider a Lyndon word u of length n (for instance, u = ab" 1 
where a < b). By definition, u is prefix of some power of a word in a necklace of 
S. Since u, as a Lyndon word, is unbordered, it cannot arise as a prefix of a proper 
power in a necklace of S. It follows that S contains a necklace of length at least n. 


If A is an alphabet of cardinality k, denote by I the set of all k! products of distinct 
elements of A: 


T= {aja2---ay|a; €A fori=1,...,k and a; a; fori F j}. 
For instance, for A = {a,b,c}, 
T = {abc,acb, bac, bca,cab, cba}. 
The following result is due to Higgins [23]. 


Theorem 8.8.7 A set S is a de Bruijn set of span n if and only if B(S) € ret 


Proof. Let us first suppose that S is a de Bruijn set of span n. Consider the matrix 
M(S). By Remark 8.8.6, the length L of the rows of M(S) is at least n. Consider the 
sub-matrix consisting of the first n columns of M(S). Since S is a de Bruijn set, the 
rows of this sub-matrix form the set A”. Each word u € A”! is prefix of k successive 
rows of M(S). We show that these successive rows of M(S) end with distinct letters of 
A. Suppose, by contradiction, that two of these rows v; and v2 end with the same letter 
a, i.e. vy = uxa and v2 = uya for some x,y € A*, with x 4 y. Since the conjugates aux 
and auy, of v; and v2, respectively, correspond to distinct rows in M(S), it follows 
that au € A” would be a prefix of a power of distinct words in the necklaces of S, 
contrary to S being a de Bruijn set of span n. Hence the final column ®(S) of M(S) 
is a product of k”~! elements (possibly with repetitions) taken from the set I. 

In order to prove the converse implication, let S be a multiset of necklaces such 
that (S) =we re" We first prove, by induction on the integer r, with 1 <r <n, 
that any word u € A* of length r is the prefix of k”~" consecutive rows of the matrix 
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M(S). In particular, we show that there exists an integer j such that u appears as a 
prefix in the rows of M(S) ranging from the index jk"~" to the index (j+1)k"""—1. 
Remark that the sequence of the last letters of these rows, read from top to bottom, 
returns a factor of w, which is again a concatenation of elements of T. 

The statement is true for r = 1. Indeed, since w € ret |w| = k” and, for any 
letter a € A, |w|q =k" !. It follows that the first column F(S) of M(S), read from top 
to bottom, consists of k”—! occurrences of the first (in the order) letter of A, followed 
by k"—! occurrences of the second letter, and so on. Actually, we have also that, if z is 
the word corresponding to an arbitrary column of M(S), for each a € A, |z|a = |z|/k. 

Let us now suppose that the statement is true for some r < n, and consider a 
word v € A* of length r +1. If a is the first letter of v, we have v = au, with |u| =r. 
By the inductive hypothesis, there exists an integer j such that u is the prefix of 
length r of k"~" consecutive rows of M(S) ranging from the index jk”~" to the index 
(j + 1)k""" — 1. The sequence of the last letters of these rows, read from top to 
bottom, forms a factor z, of w (the word corresponding to the last column of M(S)), 
and moreover z, is product of elements of I. Thus, for any a € A, |Zy|q =k" " 1, 
It follows that, within the k”~" consecutive rows of M(S) having wu as prefix, k"~"—! 
of them end with the letter a. By taking into account their conjugates, we have that 
k"-'-! consecutive rows of M(S) have as prefix the same word au = v. If b is the 
last letter of v, i.e. v = u’b, since |u'| =r, by the inductive hypothesis there exists 
an integer i such that wv’ appears as prefix of the rows of M(S) ranging from the 
index ik”~" to the index (i+ 1)k”"" — 1. The k different letters of A split the interval 
(ik”~", (i + 1)k"~" — 1] into & sub-intervals of equal length in such a way that each 
sub-interval contains the rows of M(S) having as prefix of length r+ 1 the word u'c, 
for some c € A. We conclude that there is an integer ¢ such that the k”~’—! consecutive 
rows of M(S), having as prefix the word v = u'b, have indexes that range from tk"~"~! 
to (t+ 1)k”-"-! — 1. So, we have proved that, if &(S) = w € pe , then, for any r, 
with 1 <r <n, every word u € A* of length r is the prefix of k”~" consecutive rows 
of M(S). In particular, for r =n, every word u € A* of length n is the prefix of exactly 
one row of M(S). This implies that S is a de Bruijn set of span n. a 


By Theorem 8.8.7, one can generate a de Bruijn set S of span n, on an alphabet 
A of cardinality k, by taking a word v € re" and by computing @~!(v). 


Example 8.8.8 Consider the alphabet A = {a,b} with a < b. Then T = {a B}, 
where & = ab and B = ba. Let n = 4, and consider the word v = BapBaaap = 
baabbabaabababba € T°. Rearranging the letters of v in nondecreasing order, one 
obtains the first column F(S) of the matrix M(S): F(S) = aaaaaaaabbbbbbbb. The 
inverse 7 of the standard permutation of the word v is 


—_f1 23 45 6 7 8 9 10 11 12 13 14 15 16 
Na Bsa “Bt Gd W32 AGs de Ae 8, Ge VAs AD, TA SE 


By decomposing 7 in cycles 


m = (12361159)(48 16 15 14127 13 10), 
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one obtains the set of necklaces 
S = {(aaaabab) , (aabbbbabb) }. 


One can verify that any word of A* is prefix of some word in a necklace of S, i.e. S is 
a de Bruijn set of span 4. 


Given a totally ordered alphabet A = {a,a2,...,a,}, of cardinality k, with a; < 
az <... <a, denote by @ the element a,a2---a, € T. Now we look at the special 
case of Theorem 8.8.7 where v is a power of @. In such a case, by specializing 
the arguments in the proof of Theorem 8.8.7, (cf.[23]), one can prove the following 
result. 


Theorem 8.8.9 Let v= al" ', let S =@"! (v) and let M = M(S) be the matrix cor- 
responding to S. Then the rows of M are simply the elements of A". Moreover S is the 
set of necklaces of the Lyndon words of length dividing n. 


Example 8.8.10 Consider the alphabet A = {a,b} with a < b, and the word a? = 
(ab)’. The inverse 1 of the standard permutation of the word (ab)® is 


tree eo eee eee ee 
13 5 79 It 13 15 2 4 6 8 10 12 14 #16 
By decomposing 7 in cycles 

m = (1)(23 5 9)(47 13 10)(6 11)(8 15 14 12)(16), 
one obtains the set of necklaces 


S = {(a), (aaab), (aabb), (ab), (abbb), (b)}, 


which is the set of necklaces of the Lyndon words of length dividing 4. If we consider 
the concatenation of such Lyndon words, we obtain the word 


a.aaab.aabb.ab.abbb.b 


which is indeed the first de Bruijn word of span 4 in the lexicographic order. That 
this is always the case is the well-known theorem of Frederickson and Maiorana (see 
Theorem 8.5.8). 


Actually, as a consequence of Theorem 8.8.9 and of the theorem of Frederickson 
and Maiorana, we obtain the following result. 


Proposition 8.8.11 The concatenation in ascending order of the Lyndon words of the 
necklaces of S = ®~! (a! ) is the first de Bruijn word of span n in the lexicographic 
order. 
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8.9 Suffix arrays 


Suffix array is a widely used data structure in string algorithms (see [32] or [22]). 
The suffix array of a word w of length 7 is essentially a permutation of {1,2,...,n} 
corresponding to the starting positions of all the suffixes of w sorted lexicographi- 
cally. 

Let A = {a1,d2,...,a,} be a totally ordered alphabet of size k, where a; < a2 < 
... < ag. Given a word w = 2122-:+Z, of length n on the alphabet A, the suffix ar- 
ray of w is the permutation 0,, (or simply & when no confusion arises) of the set 
{1,2,...,} such that 0 (i) = / if the suffix zjzj+1...z, has rank i in the lexicographic 
ordering of all the suffixes of w. For instance, if w = baaababa, then 


gaf te 2 245.657 8 
Tee OF as GAA I! Oey 


8.9.1 Suffix arrays and Burrows—Wheeler transform 


We show here, following [28], the close connection between the suffix array of a 
word and its Burrows—Wheeler transform. For this purpose, it is convenient to intro- 
duce the Burrows—Wheeler array (or simply the BW-array) of a primitive word of 
A*. 

Given a primitive word w = z1Z2---Z, of length n on the ordered alphabet A, 
the BW-array of w is the permutation @,, (or simply @ when no confusion arises) 
of {1,2,...,n} such that o(i) = j if the conjugate z;---zyz1---zj;-1 has rank i in 
the lexicographic sorting of all the conjugates of w. By definition, the BW-array of 
a word w is just the inverse of the permutation o defined by the relation 8.25. ie. 
g=oa!. 

In order to show the connection between the suffix array and the Burrows— 
Wheeler transform, we first introduce a sentinel symbol at the end of the word. Con- 
sider a symbol { ¢ A, and the ordered alphabet A’ = {{,a1,...,a,} where t < a, < 
... <a. We will examine the suffix array of the word w’ = wi. In the sequel, we 
denote by S,, the set of permutations of {1,2,...,2}. Moreover, for 3 € Sy, BE Sn4i 
denotes the permutation 


j= 1 2 .. ntl 
~\ n+l (1)... Bn) /)° 
Remark 8.9.1 There is a one-to-one correspondence between the suffix arrays of 
the words w € A” and the suffix arrays of the words w’ € A”. In particular, if the 


permutation %,, € S, is the suffix array of w € A”, then the permutation 0, € S,+1 is 
the suffix array of w’ = wit if and only if 3, = 3. 


Remark 8.9.2 It is easy to see that, for words in A*f, conjugate sorting is equivalent 
to suffix sorting. It follows that the suffix array of the word w’ = wi coincides with 
its BW-array, ie. = Oy. 
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The following statement follows from the previous remarks. 


Proposition 8.9.3 A permutation 0 € Sy is the suffix array of a word w © A” if and 
only if the permutation 9 € Sy, is the BW-array of the word w’ = wit. 


Consider now the mapping W : S, — S,+1 defined as follows. If 0 € S,, ¥(d) 
is the permutation 1 € S,41 defined by u(i) = b~'(8(i) +1), where the addition is 
taken modulo n+ 1. Actually, Y(®) is just the permutation obtained by writing 3—! 


as a word and interpreting it as a (n+ 1)-cycle. 


Example 8.9.4 Consider the permutation 


Te 3. Aes 
Crees 


ony 
—_— ~] 
WM CO 
eee 


Then we have that 


and 


It follows that 
123 45 67 8 9 
¥(8) = (834695721) =( 14679 2 3 a 


The following theorem, that appears in [28], gives a characterization of suffix 
arrays. 


Theorem 8.9.5 Let nj,n2,...,nx be positive integers such that nj +no+...+ng = 
n. A permutation 0 € Sy, is the suffix array of a word w € A”, with Parikh vector 
P(w) = (n1,n2,...,nx) ifand only if Des(¥(Y)) C {1,l+m,...,l+aj+...+ng_1}- 
Moreover, in this case, © is the suffix array of exactly one such word. 


Proof. Let us suppose that 0 € S, is the suffix array of a word w € A”. Then, by 
Proposition 8.9.3, 3 € S,1 is the BW-array of the word w’ = wi. By observing that 
the BW-array of w’ corresponds to the inverse of the permutation o,, defined by the 
relation (8.25), we have that 3 = (o,,)~!. We show that ‘¥() = m,, where 7, is 
the permutation defined by formula (8.28). Indeed, if u = ‘Y¥(), we can write 


u(i) = 818) +1) =o (oy () +1) = my. 


We now observe that, if P(w) = (1,...,,) is the Parikh vector of w, then the Parikh 
vector of w’ = wi is P(w’) = (1,71,...,nx). Therefore, by Theorem 8.7.7, 


Des(¥(8)) = Des(m,) C p(P(w’)) = {1,1 +m,..., l+n)+...+mp_1}. 
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Conversely, given a Parikh vector V = (n1,...,n,) and a permutation 0 € S,, such that 
Des(¥(8)) C {1,1 +m1,..., 1 +1 +...-+-ng_1}, we show that there exists a unique 
word w € A” having Parikh vector P(w) = V and suffix array 0, = 0. Actually, we 
provide a construction of this word. Since a, < a2 < ... < dx, in the starting positions 
of the first n; suffixes of w, in the lexicographic order, there is the letter a;, in the 
starting positions of the suffixes of w having rank from n; + 1 to ny +72, in the 
lexicographic order, there is the letter az, and so on. Therefore, for w = z1Z2-++Zn, if 
1l<i<n then Z9(j) =a, and, forl <r<k,ifny+...+n,_| <i<njt+...+n,, 
then z(;) = a,;. This concludes the proof. | 


Example 8.9.6 Given the permutation 0 € Sg in Example 8.9.4 and the vector V = 
(5,3), we construct a word w on a binary alphabet A = {a,b}, with a < b, having V 
as Parikh vector and © as suffix array. From Example 8.9.4, we have that 


Des(Y(&)) = {1,6} verifies the condition of the Theorem 8.9.5. The word w = 21...8 
having Parikh vector (5,3) and suffix array 8 is obtained as follows: 


Z9(1) = <9(2) = %0(3) = %9(4) = 205) = 4 


and 


29(6) = 29(7) = 20(8) =F. 


Therefore w = baaababa. 
The following corollary of Theorem 8.9.5 will be useful in the next section. 


Proposition 8.9.7 A permutation 0 € S,, is the suffix array of some word w of length 
n onan alphabet of cardinality k if and only if 


Card(Des(¥(8)) \ {1}) <k-1. 


8.9.2 Counting suffix arrays 


The results of previous sections are used here to solve three enumeration problems 
concerning suffix arrays. The results are essentially due to Schurmann and Stoye [38] 
(see also [15], [3], and [28]). 

The first problem approached here is to count the number s(n,k) of distinct per- 
mutations that are suffix arrays of some word of length 7 over an alphabet of size 
k. 

The following table gives the values of s(n,k) for2 <<k<n<9. 
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12 23 24 

93 119 120 

58 360 662 719 720 

121 1312 3728 4919 5039 5040 

248 4541 20160 35779 40072 40319 40320 

503 15111 103345 259535 347769 362377 362879 362880 


OMAADNHWNS & 
N 
= 


In the next theorem we show that the function s(n,k) is related to the Eulerian 
numbers ays i.e. the number of permutations of {1,2,...,2} with exactly d descents. 
Recall (cf.[21]) that the Eulerian numbers can be defined by the following recurrence 


relation ; ; 
n n— n— 
— 1 _— 
Cg EN) 
with (5) = 1 and ()) =0 whend > 1. 


Theorem 8.9.8 The number s(n,k) of distinct permutations that are suffix arrays of 
some word of length n over an alphabet of size k is 


s(n,k) = y e 


d=0 


In order to prove the theorem we need a preliminary lemma. In the following it 
is convenient to represent a permutation @ € S, by the word @(1)@(2)...@() on the 
alphabet {1,2,...,2}. Now we define a mapping that, for any @ € S, and for any 
s € {2,3,...,n+1}, gives a permutation y € S,,1. Such a mapping is described as 
a transformation on words performed in three steps. 

For a permutation 9(1)9(2)---@(n) and an integer s € {2,3,...,2 +1}, in the 
first step we obtain the word 


Es(@) = @s(1)@s(2)-+- @s(n), 


where @;(i) = (i) for @(i) < s, and 9,(i) = @(i) +1 for @(i) > s. Remark that 
@;(1)@,(2)--+@5(”) is a word on the alphabet {1,2,...,n,n+ 1}, but it does not rep- 
resent a permutation, because the integer s does not appear in the word. For instance, 
consider the permutation @ € S¢ represented by the word 364215 and s = 3. Then 
E3(@) = 475216. 

In the second step J;, which is the most important, we move @,(1) from the first 
position in the word to the position s — 1. It is called the insertion step and it is 
formally defined as follows: 


Ts(Ps(1) Ps (2) ++ Ps(7)) = Ps(2) > Ps(5 — 1) Ps(1) @s(S) +++ Ps (72). 
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For instance, 1;(475216) = 745216. 

In the third step C; we simply insert the symbol s in the first position of the word. 
For instance, C3(745216) = 3745216. 

The compositions of the above operations define the transformation T(@,s) = 
Cs(Us(Es(@))). Remark that the word T(@,s) represents a permutation of 
{1,2,...,n,n +1}. For instance, for @ = 364215 and s = 3, we have T(@,5) = 
3745216. Moreover, it is straightforward to check that, if @ is cyclic, then T(9@,s) 
is cyclic too. Therefore, if we denote by St the set of cyclic permutations of 
{1,2,...,n}, the transformation T defines a mapping 


T : Six {2,3,...,.2+1} > Shi. 


Lemma 8.9.9 The mapping T is a bijection from Sy, x {2,3,...,n+ 1} onto St. 


Proof. We first prove that T is injective by showing that, given a permutation y € 
S*, 1, one can uniquely reconstruct the pair (~,s), with gp € Sf ands € {2,...,n+]}, 
such that T(@,s) = yw. Let wy = w(1)w(2)---w(n+ 1). Since y is a cyclic per- 
mutation, y(1) 4 1. By the definition of T, s = w(1). We delete w(1) = s from 
the word y(1)w(2)---w(n+ 1), and we obtain the word y(2)---w(n+ 1). Then 
we take the element y(s) and move this element in the first position of the word. 
We obtain the word w(s)y(2)--- w(s — 1)w(s+1)---w(n +1). Now we substi- 
tute each y(j) > s with w(j) — 1 and we obtain the permutation @ € S¢ such that 
T(@,s) = w. In order to show that the mapping T is surjective, it suffices to verify 
that Card(S;, x {2,3,...,2+1}) =Card(S¢, |). Indeed Card(S;, x {2,3,...,2+1}) = 
(n—1)!n =n! = Card(S¢,, ;). | 
Proof of Theorem 8.9.8. | According to Proposition 8.9.7, there is a bijection be- 
tween the suffix arrays of words w € A" and the cyclic permutations y € S’,, such 
that Card(Des(y) \ {1}) < k-— 1. We have then to count the number of such permu- 
tations. 

Let P(n,d) denote the number of permutations y € S’_, , such that Card(Des(y) \ 
{1}) =d. To prove the theorem, we show that P(n,d) is equal to the Eulerian number 


(a): 

The proof is by induction on n. Trivially, P(1,0) = 1 = (}), and P(1,d) =0 = (*) 
whend > 1. 

We now show that P(n,d) = (d+ 1)P(n—1,d)+(n—d)P(n—1,d—-1). 

By Lemma 8.9.9, a permutation y € S’,, , can be obtained, through the transform 
T, from a permutation @ € S¢ with the insertion of an element s € {2,...,1+ 1}. 
We now examine how the transform T affects the number of descents of @. Remark 
that the steps 1 and 3 in the definitions of the transform T do not affect the number 
of descents. This number can be affected only in step 2 (the insertion step /,). If @ 
has d descents in the interval {2,...,2-+1}, also Es(@), the word obtained after the 
first step, has d descents, independently from the choice of s. We can thus factorize 
E;(@) ind + 1 monotonic (increasing) runs. The second step in the transform T (the 
insertion step J,;) may or may not create a new descent, depending on the position in 
which is inserted the first symbol @,(1) of the word E,(@). In each monotonic run 
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of E;(@) there is exactly one position where @,(1) can be placed without creating a 
new descent. Otherwise one creates exactly one new descent. 

How many permutations y = T(@,s) can we obtain with Card(Des(w) \ {1}) = 
d ? For each @ € S¢ with Card(Des(@) \ {1}) =d, we have d+ 1 possibilities to 
choose s (because in Es(@) there are d+ 1 monotonic runs). For each @ € S¢ with 
Card(Des(@) \ {1}) =d—1, we have n —d possibilities to choose s. Since T is a 
bijection, there is no other way to get a permutation y € S’,, with Card(Des(y) \ 
{1}) =d. It follows that 


P(n,d) = (d+ 1)P(n—1,d)+(n—d)P(n—1,d—-1). : 


We now consider the problem of counting the number of words that share the 
same suffix array. 


Theorem 8.9.10 Given a permutation 3 € S,, the number of words of length n over 
an alphabet of size k having © as their suffix array is 


n+k—1—d 
( pa ), 
where d = Card(Des((#)) \ {1}). 


Proof. By Theorem 8.9.5, a word w € A”, with |A| =k, has 0 as suffix array if and 
only if w has a Parikh vector P(w) = (m1,n2,...,”,) such that 


Des(¥(8)) C {1,1 +7,...,l +a +...+ng_1}. 
Therefore, given the permutation 0, and then given the set 
Ds = Des(¥(8)) \ {1} = {11,m2,...,ma}, 
we need to count the number of tuples (7,...,7,), with ny +...+-ny, =n such that 
Dg Cf{ltny,ltnytm,...,l+tnypt...+ng_y}. 
We represent the tuple (11,...,,) by a word z on the alphabet {x, y}: 
Z = Xilygy.. x LyxM, 


with n; > 0 and nj +...+ny =n. We have that |z|] =n+k-—1. The condition 
Ds = {m,...,ma} C {1 4+m1,...,1 +21 +...+ng_1} defines the positions of d 
occurrences of the letter y in z. The remaining k — 1 —d occurrences of y can be 
placed in arbitrary positions. This can be done in 


n+k—1-—d 
k—1-d 


ways. a 
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Note that if k— 1 < Card(Des(¥(#)) \ {1}), there is no word on an alphabet 
of size k that has ¥ as its suffix array. This is confirmed by Theorem 8.9.10, since 
(”") =0 form <n. 

In the next theorem, we require that each letter of the alphabet occurs at least 
once in the words that we count. 


Theorem 8.9.11 Given a permutation 0 © Sp, the number of words of length n over 
an alphabet of size k that have at least one occurrence of each of the k letters and 


have © as their suffix array is 
n—1-—d 
k—1-—d/)’ 


where d = Card(Des(¥(#)) \ {1}). 


Proof. The proof of Theorem 8.9.10 is modified in order to ensure that each letter 
occurs at least once. In the representation of the tuple (m1,...,n,) by the word z = 
XMyx"2y-..x7k-lyx"k, we require that the n; are strictly positive, i.e. nj > 0 for i= 
1,...,4—1. Then we have to distribute the occurrences of the letter y among the n — 1 
possible positions. As in the proof of Theorem 8.9.10, the positions of d occurrences 
of y are determined by the permutation 0, and the remaining k — 1 —d are distributed 
among the n — 1 — d remaining positions. a 


From Theorem 8.9.8 and Theorem 8.9.10 we can derive a long known summation 
identity of Eulerian numbers. The identity 


ee ECT) 


as given in [21, Eq.6.37], was proven by J. Worpitzki in 1883. In order to prove it, 
we observe that the number of words of length n over an alphabet of size k can be 
obtained by summing the number of words for each suffix array. Thus, we have: 


e-y n\ (n+k—d-1 
— At \d/\ k-d-1 J 


By using the symmetry rule for Eulerian and binomial numbers, from the previous 


equality we derive 
pet y n n+k—d-1 
- joy \ 1 -d n 


By setting 7 =n —d-—1, we obtain 


Pe era) 


where the last equality is motivated by the remark that (“) = 0 for all 7 >n and 
(‘*/) =0 forall j <n—k. 
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9.1 Introduction 
We begin with some examples. 


Example 9.1.1 Figure 9.1 shows a region (a 2-by-6 rectangle) that has been tiled by 
dominos (/-by-2 rectangles) in one of 13 possible ways. More generally, the number 
of domino tilings of the 2-by-n rectangle (for n > 1) is the nth Fibonacci number (if 
one begins the sequence 1,2,3,5,8,...). 


541 


542 Handbook of Enumerative Combinatorics 


Figure 9.1 
A domino tiling (one of 13). 


Example 9.1.2 Figure 9.2 shows a region, composed of 24 equilateral triangles of 
side-length 1, that has been tiled by lozenges (unit rhombuses with internal angles 
of 60 and 120 degrees) in one of 14 possible ways. The region shown in Figure 9.2 
is part of a one-parameter family of regions whose nth member has (7") /(n +1) 
tilings by lozenges. (See [47] and [51] for more on this manifestation of the Catalan 
numbers. ) 


Figure 9.2 
A lozenge tiling (one of 14). 


Examples 9.1.1 and 9.1.2 are prototypes of the sorts of problems discussed in this 
chapter. 

Specifically, we have some compact region R in the plane (in Example 9.1.1, R 
is the 2-by-6 rectangle [0,6] x [0,2]) and a finite collection of other compact subsets 
called prototiles (in Example 9.1.1, the prototiles are [0, 1] x [0,2] and [0,2] x [0, 1]), 
and we are looking at unordered collections {S),...,5;} such that (a) each S; is a 
translate of a prototile, (b) the interiors of the S;’s are pairwise disjoint, and (c) the 
union of the S;’s is R. We call such a collection a tiling of R. Given a region to be 
tiled and a set of prototiles, we want to know: In how many different ways can the 
region be tiled? 

What concerns us here are not individual tiling problems but infinite families of 
tiling problems, with each family determining not just one number but an infinite 
sequence of them, indexed by a size parameter we will typically call n. The set of 
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prototiles remains the same, while the size of R goes to infinity with n. In some cases, 
we obtain a sequence that is governed by a closed-form formula or a recurrence rela- 
tion; in other cases, we settle for asymptotics. Often we find that the individual terms 
of the sequence exhibit number-theoretic patterns (e.g., they are perfect squares or 
satisfy certain congruences) that call out for a theoretical explanation, even in the 
absence of a general formula for the terms. The article [137] contains many such 
problems, as well as discussions of general issues in enumeration of tilings that over- 
lap with the discussions contained in this chapter. 
Here are two more examples of such one-parameter tiling problems: 


Example 9.1.3 (Thurston and Lagarias-Romano) Figure 9.3 shows a “honey- 
comb triangle” of order 10 with one hexagon omitted (the central hexagon), tiled 
by 18 regions called tribones. W. Thurston [166] conjectured that for each way of 
packing 3n(n+ 1)/2 tribones into a honeycomb triangle of order 3n + 1, the unique 
uncovered cell will be the center cell. Lagarias and Romano [107] proved this con- 
jecture and also showed that the number of such packings is 2”. 


ae 
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Figure 9.3 
A tribone tiling (one of 8). 


Example 9.1.4 (Moore) Figure 9.4 shows a 5-by-5 square tiled by ribbon tiles of 
order 5, where a ribbon tile of order n is a union of n successive unit squares, each 
of which is either the rightward neighbor or the upward neighbor of its predecessor. It 
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is an amusing puzzle to show that the number of tilings of the n-by-n square by ribbon 
tiles of order n is n\. (The proof is not hard, but it is not obvious.) This unpublished 
example was communicated to me by Cris Moore. 


Figure 9.4 
A ribbon tiling (one of 120). 


These last two examples are pretty, but they are not typical of what you will learn 
about in this chapter (outside of the first part of Section 9.2). First, they do not serve 
as examples of a general method. Second, the sequences one gets grow too slowly 
to be typical of the state of the art in the modern theory of enumeration of tilings. 
Calling n! “slow-growing” may seem surprising, since it grows faster than any expo- 
nential function. However, one thing that the nth Fibonacci number, the nth Catalan 
number, n!, and 2” have in common is that their logarithms grow linearly in n (at least 
up to a factor of logn or smaller). In contrast, the theory of enumeration of tilings 
shows its true strength with exact enumerative results involving functions of n whose 
logarithms grow like n*. A prototype of theorems of this kind concerns regions called 
Aztec diamonds [53]. An Aztec diamond of order 7 consists of 2n centered rows of 
unit squares, of respective lengths 2,4,...,2n —2,2n,2n,2n —2,...,4,2. Figure 9.5 
shows an Aztec diamond of order 5 tiled by dominos in one of the 32,768 possible 
ways. More generally, an Aztec diamond of order n has exactly one tilings by 
dominos. This result has now been proved a dozen different ways, though some of 
the differences are more cosmetic than conceptual. 

Highly analogous to domino tilings of Aztec diamonds are lozenge tilings of 
hexagons. Call a hexagon semiregular if its internal angles are 120 degrees and op- 
posite sides are of equal length (more generally, call a polygon with an even number 
of sides semiregular if opposite sides are parallel and of equal length). A semiregular 
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Figure 9.5 
A domino tiling of an Aztec diamond (one of 32,768). 


hexagon with side-lengths a,b,c,a,b,c can be tiled by lozenges in exactly 


H(a+b+c)H(a)H(b)H(c) 
H(a+b)H(a+c)H(b+c) 


ways, where H(0) = H(1) = 1 and H(n) = 1!2!---(n—1)! for n > 1; an equivalent 
expression is 
pt ptk—1 
TW y= 


it jalk=1 bt JT 


This result goes back to [119], although MacMahon was not studying tilings but 
plane partitions; plane partitions whose parts fit inside an a-by-b rectangle and with 
all parts less than or equal to c have three-dimensional Young diagrams that are 
easily seen to be in bijection with lozenge tilings of the a,b,c,a,b,c semiregular 
hexagon [156]. Figure 9.6 shows a regular hexagon of order 4, tiled by lozenges in 
one of the 232,848 possible ways. 

Could there be higher-dimensional analogues of these tiling problems, involving 
sequences whose logarithms grow like n> (or faster)? Curiously, such analogues are 
almost entirely lacking (and I will briefly return to this issue later, in a speculative 
vein, at the close of the chapter). For now, I wish only to point out that any theory of 
exact enumeration of tilings that attempts to catch too many tiling problems in its net 
is likely to find that net ripped apart by the sea monsters of #P-completeness. (For a 
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Figure 9.6 
A lozenge tiling of a hexagon (one of 232,848). 


definition of #P-completeness, see [125].) Hardly anyone believes that #P-complete 
problems can be solved efficiently. Beauquier, Nivat, Rémila, and Robson [6], Moore 
and Robson [124], and Pak and Yang [129, 130] have given examples of classes 
of two-dimensional tiling problems exhibiting #P-completeness. So there is little 
hope of solving the problem of counting tilings in its full generality, even in two 
dimensions. Still, there is much that can be done. 

By far the most successful theory of enumeration of two-dimensional tilings is 
the theory of perfect matchings of a planar graph. A perfect matching of a graph 
G = (V,E) is a subset E’ of the edge-set E with the property that each vertex v € V is 
an endpoint of one and only one edge e € E’. Problems involving perfect matchings 
of a planar graph G may not immediately appear to be tiling problems, but they can 
easily be recast in this framework: simply take the planar dual of G (the planar graph 
whose vertices, edges, and faces correspond respectively to the faces, edges, and 
vertices of G) and define a tile to be the union of any two regions in the dual graph 
that correspond to adjacent vertices in G. (To make this correspondence precise, one 
may need to limit the kinds of translation that can be applied to the prototiles.) In 
the other direction, some tilings correspond to perfect matchings. For example, the 
domino tiling of Figure 9.5 corresponds to the perfect matching in Figure 9.7. For 
a more careful definition of Aztec diamonds and their dual graphs, see [53]. See 
also Figure 1 from [135], which shows the relationship between rhombus tilings of 
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a hexagon and dual perfect matchings, and Figure 2 from [135], which shows the 
relationship between domino tilings of a square and dual perfect matchings. 


DLL 
see oe 


Figure 9.7 
A matching of an Aztec diamond graph (one of 32,768). 


(In the graph theory literature, a matching of a graph G is a subset E’ of the edge- 
set E with the property that each vertex v € V is an endpoint of at most one edge 
e € E’. I will comment later on the problem of counting matchings, or as physicists 
call them, monomer-dimer covers. Until then, I will use the term “matching” to refer 
exclusively to perfect matchings.) 

The problem of counting matchings of a general graph is #P-complete [168], but 
when the graph is planar (or not too far from being planar), the problem of counting 
matchings can be reduced to linear algebra, specifically, to the evaluation of deter- 
minants (and Pfaffians) of matrices. This technology was developed by the mathe- 
matical physicist Kasteleyn, who (using physicists’ language) thought of his work 
as providing a way to evaluate “the partition function of the dimer model” (we will 
explain that language later). Thanks to Kasteleyn’s work [78] [79] [80], much of the 
known body of theory relating to enumeration of tilings in the plane can be viewed as 
a subspecialty of the field of determinant evaluation, as developed by Krattenthaler 
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and others (see for instance [93] and [95]). However, in many instances the matrices 
that arise do not belong to a general class of matrices to which established methods 
of determinant evaluation apply; in those instances, the only known ways of evaluat- 
ing the determinant make crucial use of the combinatorics and geometry of the tiling 
picture. 

The main models I will treat are domino-tilings and lozenge-tilings, and the main 
methods of analysis I plan to discuss (in varying levels of detail) are: the trans- 
fer matrix method (Section 9.2); the Lindstr6m-Gessel-Viennot method, Kasteleyn- 
Percus method, and spanning tree method (Section 9.3); representation-theory meth- 
ods (Section 9.4); and Ciucu complementation and factorization, Kuo condensation, 
and domino shuffling (Section 9.5). In Section 9.6, I will try to give a sense of the 
broader context into which the enumeration of tilings fits, and convey some sort of 
historical sense of the way in which the field has evolved (or at least point the his- 
torically minded reader toward earlier articles from which an accurate history of the 
subject could be assembled). It should be stressed that many of the tools discussed 
in Sections 9.2 through 9.5 were developed not by combinatorialists but probabilists, 
chemists, and physicists; a knowledge of the broader scientific literature has served 
combinatorialists well in the past and is likely to do so in the future. 

In Section 9.7, I will discuss themes that have emerged from the study of match- 
ings of planar graphs such as special patterns of edge-weights that, although non- 
periodic, seem in some sense to be natural and in any case give rise to nice g- 
analogues of integer enumerations; three distinct ways in which the concept of sym- 
metry plays a role; and certain sorts of patterns that often appear in the prime factor- 
ization of the integers that arise from enumeration of matchings. 

In Section 9.8, I will describe some of the software that exists for exploring prob- 
lems in enumeration of matchings (sadly in disuse and disrepair, but that surely will 
change if there is sufficient new activity in this field to justify investment in infras- 
tructure). In Section 9.9, I will give an admittedly personal and biased list of issues 
on which progress is needed. Unlike [137], which focused on specific problems in 
enumeration, the current list is about larger themes. 

My aim will be not to provide definitive and general statements of results, but 
to give attractive examples that will make combinatorially-inclined readers want to 
learn more about (and perhaps work in) this area, and will help researchers starting 
out in this area get a better sense of what they might want to read. 

For a recent article on tilings by other authors, see [2]. 


9.2 The transfer matrix method 


Looking down each of the six columns in Figure 9.1, we see either a vertical tile, the 
left halves of two horizontal tiles, or the right halves of two horizontal tiles. More- 
over, if we know, for each of the six columns, which of these three pictures we see 
(Vertical, Left, or Right), then we know the whole tiling. For the tiling shown in Fig- 
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ure 9.1, the “code” is VERVLR. However, not every length-six sequence consisting 
of V’s, L’s, and R’s corresponds to an allowed tiling. The codes that correspond to 
tilings are precisely the ones that arise from paths of length six in the graph shown in 
Figure 9.8, starting at the circled node on the left and ending at the circled node on 
the right, where the symbols V, L, and R correspond respectively to edge of slope 0, 
—1, and +1. For instance, the path shown in bold has successive edges of type V, L, 
R, V, L, and R, and therefore corresponds to the tiling of Figure 9.1. 


Figure 9.8 
A trellis graph for counting domino tilings. 


Figure 9.8 is an example of what I will call a trellis graph (though my use of 
the term is slightly different from the standard definition): a graph with vertex set 
Vo UVa U---UV,, where each V; (0 < k <n) is finite and all edges connect a vertex in 
V,_1 to a vertex in V; for some 1 <k <n. For each 1 <k <n, let M; be the matrix 
with |V;_;| rows and |V;| columns whose i, jth entry counts the edges from the ith 
element of Vi_; to the jth element of V;. Then it is easy to see that the number of 
paths from the ith element of Vo to the jth element of V,, equals the 7, jth entry in 
the matrix product M,|M)---M,. We call the M;’s transfer matrices. In our case, all 
1 1 
1 0 
number of tilings as the upper-left entry of M°, which is 13. More generally, we see 
that the number of domino-tilings of the 2-by-n rectangle is the upper-left entry of 
M", which is the nth Fibonacci number. 

The same method can be applied to count domino-tiling of an m-by-n rectangle. 
The number of rows and columns in the matrix M grows exponentially with m, but 
Theorem 4.7.2 of [159] assures us that for fixed m, the number of domino-tilings of 
an m-by-n rectangle, viewed as a function of n, satisfies a linear recurrence relation 
with constant coefficients. 

This method can be applied much more broadly to tiling problems in which the 
region being tiled is “essentially one-dimensional” in the sense that only one dimen- 
sion is growing as n increases. For example, if a, denotes the number of ways to 
pack an 8-by-8-by-n box with 1-by-1-by-2 bricks that can be used in each of the 
three possible orientations, then the general theory tells us that a, must satisfy a lin- 
ear recurrence. However, the span of this recurrence could be extremely long; an 
off-the-cuff upper bound on the span is 10°°, which can probably be improved a bit 
but not by much without a good deal of work. 

By using matrices whose entries are not just 0’s and 1’s, we can get more mileage 
out of the method. Suppose for instance we want to discriminate between tilings on 


the matrices are the same matrix M = ( ), so we can readily compute the 
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the basis of how many horizontal or vertical dominos they have. Then we would 
look at powers of the matrix M = ’ 0 
domino and each L or R corresponds to two horizontal half-dominos). The upper-left 
entry of M® is x° + 6x+y* + 5x?y4 + y°, which tells us that the tiling of Figure 9.1 is 
one of exactly six tilings having four horizontal dominos and two vertical dominos. 

Typically one exploits the transfer-matrix method by diagonalizing M (and if one 
is only interested in the growth rate of the entries of M”, one merely computes the 
dominant eigenvalue of M, invoking the Perron-Frobenius theorem to conclude that 
the dominant eigenvalue is a simple eigenvalue of the matrix). 

In rare cases, one can actually get formulas for the entries of the powers of M and 
skip the diagonalization step. One way of counting the lozenge tilings of semiregular 
hexagons provides an example of this: see Proposition 2.1 of [41] and its proof. 
One can argue that such a proof does not deserve to be called an example of the 
transfer-matrix method, since no linear algebra is involved. (Perhaps it should be 
called “enumeration by exact dynamic programming” or “the method of proving a 
result by finding a stronger result that can be proved by induction.’’) 

We conclude by returning to the x, y-enumeration of domino tilings discussed 
above, putting it in the framework of statistical mechanics, and indicating how the 
transfer matrix method can be applied to all one-dimensional lattice models. We 
already mentioned in Section 9.1 that domino tilings of a region are in bijection 
with matchings of the graph dual to the tessellation of the region by unit squares. 
In physics, a (perfect) matching is called a dimer cover or dimer configuration. We 
assign each dimer cover a weight equal to the product of the weights of its constituent 
edges, where a horizontal (respectively, vertical) edge has weight x (respectively, y). 
In physics, the weights have an interpretation in terms of energy (see [5]) but we will 
not concern ourselves with that here. The sum of the weights of the configurations 
is called the partition function, and is denoted by Z. The probability measure that 
assigns to each configuration probability equal to the weight of the configuration 
divided by the normalizing constant Z is called the Gibbs measure associated with 
that weighting. In the case where all configurations have the same weight, and the 
probability distribution is correspondingly uniform, we define the entropy of the 
system as the logarithm of the number of (equally likely) states. More generally, 
if a system has N states with respective probabilities p;, we define the entropy as 
yy pjlog 1/p;, which coincides with the former definition if pj = 1/N for 1 <i<N. 

These concepts, with some work, can be extended to infinite systems as well, 
through appropriate rescalings and limit-procedures. Some care is helpful for deal- 
ing with the boundary conditions, but these turn out not to matter so much in one 
dimension. For instance, in the case of the 2-by-2n dimer model with edge-weights 
x and y (which we take to be positive), we might take the logarithm of the number 
of tilings of the 2-by-n rectangle, divide by the area 2n, and take the limit as n — ~, 
obtaining the normalized, or per-site, entropy of the model, a function of x and y. Or 
we might look at 2-by-n configurations in which dominos are allowed to protrude at 
the ends. The same asymptotics will prevail, since the entries of M” must all grow at 
the same rate. 


x : F 
) (since each V corresponds to a vertical 
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This argument is not applicable to higher-dimensional tilings, and indeed, the 
per-site entropy turns out to be extremely sensitive to the boundary conditions. For 
example, if one modifies the Aztec diamond of order n so that it has only one row of 
length 2n instead of two, the number of tilings drops from 2n(n+1)/2 down to 1. 

In the case where all the states of a one-dimensional lattice model have the same 
probability, the transfer matrix method tells us that the entropy is just the logarithm 
of the dominant eigenvalue. In particular, for one-dimensional lattice models of this 
kind, if all the entries of the transfer matrix are 0’s and 1’s, the entropy must be the 
logarithm of an algebraic number. This, too, fails in higher dimensions (at least as far 
as is known); although the entropy for the square ice model introduced in Section 9.4 
is the logarithm of the algebraic number 8 / (3/3), the entropy of the dimer model on 
a square grid to be discussed in Section 9.3 is G/a (where G is Catalan’s constant), 
which is not believed to be the logarithm of an algebraic number. 


9.3. Other determinant methods 
9.3.1 The path method 


Returning to Example 9.1.1 of Section 9.1, let us decorate each tile according to its 
orientation in the fashion shown in Figure 9.9, so that Figure 9.2, with all its tiles 
decorated, becomes Figure 9.10. 


Figure 9.9 
Decorated lozenges. 


The decorations on the tiles yield a path from the leftmost node (marked P) to the 
rightmost node (marked Q), along with some nodes that are not visited by the path. 
In this way, one can see that the tilings of the region are in bijection with Dyck paths 
of length 6, and hence are enumerated by the Catalan number 14. 

Figure 9.11 shows a domino tiling of a 3-by-4 rectangle decorated with a similar 
sort of path. This Randall path [116] is obtained by drawing a node at the midpoint 
of every other vertical edge (that is, when two vertical edges are neighbors in either 
the horizontal or vertical sense, exactly one of them has a node). Every tile then either 
has a single node in its interior or two nodes on its boundary, and in the latter case 
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Figure 9.10 
From tilings to paths. 
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Figure 9.11 
A tiling and a path. 


we join the nodes by a line segment. All the edges in the path go from left to right, 
with slope 0, +1, or —1. It is easy to use dynamic programming to show that there 
are 11 paths from P to Q, and hence 11 domino tilings of the rectangle, since there is 
a bijection between tilings and paths. 

Figure 9.12 shows the other way to decorate the tiling of Figure 9.11 by drawing 
a node on every other vertical edge. Now we get not just one path but two noninter- 
secting paths, one joining P; to Q; and one joining P; to Qo. There are six paths from 
P; to Q; when i = j but only five when i 4 j. Lindstrém’s lemma [113], rediscovered 
by Gessel and Viennot [65], shows that the number of pairs of nonintersecting paths 
from P; and P to Q; and Q? respectively equals the determinant of the 2-by-2 matrix 
whose i, jth entry counts the paths from P; to Q;. Thus the number of tilings is 


6 5 , 
| 5 6 [= 11 Gaim 


since there is a bijection between tilings and pairs of nonintersecting paths. 
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Figure 9.12 
A tiling and two nonintersecting paths. 


This method generalizes to larger collections of paths in an acyclic directed 
graph. As long as the only way to join up the points P,,...,P, to the points 
Q\,...,Qm by nonintersecting paths is to join P; to Q; for all | <i<m, the num- 
ber of such families of nonintersecting paths equals the determinant of the matrix 
whose i, jth entry is the number of paths from P; to Q;. In many cases one can use 
this to count matchings of graphs. By employing this method, which was first used 
in [67] and was also discovered by chemists John and Sachs [77], one can write the 
number of lozenge tilings of a semiregular hexagon as the determinant of a matrix 
of binomial coefficients [65] (see also [19]); likewise one can write the number of 
domino tilings of an Aztec diamond as the determinant of a matrix of Schréder num- 
bers [17, 55]. A recent (and purely bijective) approach to counting nonintersecting 
Schroder paths is [13]. For historical background on Lindstrém’s lemma, see foot- 
note 5 in [96] as well as the Notes to Chapter 2 of [159]. 


9.3.2. The permanent-determinant and Hafnian-Pfaffian method 


Consider the 3-by-4 grid graph (dual to the 3-by-4 rectangle considered in the previ- 
ous subsection) and two-color the vertices so that each black vertex has only white 
neighbors and vice versa. Number the white vertices 1 through 6 and the black ver- 
tices | through 6. The ordinary 12-by-12 adjacency matrix of this graph has deter- 
minant 0, as does the 6-by-6 bipartite adjacency matrix whose i, jth entry is 1 or 0 
according to whether the ith white vertex and jth black vertex are connected or not. 
However, if we let K be the modified bipartite adjacency matrix whose i, jth entry 
is l,i(= J/—-1 ), or O according to whether the ith white vertex and jth black vertex 
are connected by a horizontal edge, a vertical edge, or no edge, then we find that 
the modulus of the determinant of K equals the number of matchings of the graph. 
Likewise, if we replace the |’s in the original 12-by-12 adjacency matrix with suit- 
able unit complex numbers (complex numbers of modulus 1), we obtain a matrix 
whose determinant has modulus equal to the square of the number of matchings of 
the graph. 
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This is a consequence of a general theorem of Kasteleyn. We discuss first the case 
of bipartite graphs (tacit in Kasteleyn’s original papers [78, 79, 80] and made explicit 
by Percus [131]). If a bipartite graph G with 2N vertices is planar, then the non-zero 
entries of its bipartite adjacency matrix can be replaced by unit complex numbers 
so that the modulus of the determinant of the resulting modified matrix equals the 
permanent of the original matrix and hence equals the number of perfect matchings 
of G. More specifically, let us say (using terminology introduced by Kuperberg) that 
an edge-weighting of a bipartite planar graph that assigns each edge a unit complex 
number is (Kasteleyn-)flat if the alternating product of the edge-weights around a 
face efleslettes! --» is —1 or +1 according to whether the number of edges is 0 or 
2 mod 4 (this condition does not depend on the cyclic ordering of the faces). In our 
case, each face has four sides, so we need the alternating product to be —1, and indeed 
we have (i)+!(1)~!(i)+!(1)~! = —1. (The fact that flatness holds around the exterior 
face is a consequence of flatness around the interior faces; one can also check directly 
that, in the case of the 3-by-4 grid graph, the alternating product around the 14-sided 
exterior face is i /i? = +1.) Let K be the bipartite adjacency matrix of a bipartite 
planar graph G, with non-zero entries replaced by Kasteleyn-flat edge weights. It can 
be shown that if one expands the determinant of K into NV! terms then all the non-zero 
terms in the expansion are unit complex numbers with the same phase. Consequently, 
the matchings of the graph can be counted by taking the modulus of the determinant 
of the modified adjacency matrix obtained from the flat edge-weighting. This is the 
permanent-determinant method. 

To see this method applied to general rectangular subgraphs of the square grid, 
see [133]. It can be shown that the per-site entropy of the dimer model on a square 
grid is G/2m, where G = Y"~}(—1)"/(2n+1)? =1/1—-1/9+1/25—... (Catalan’s 
constant). 

Now suppose G is a non-necessarily-bipartite planar graph with 2N vertices. Re- 
call from [101] that the Pfaffian of an antisymmetric 2N-by-2N matrix A equals a 
signed sum of terms of the form aj, j, Gi,,j,°**Giy,jy Where i), j1,...,in, JN Tanges 
over all permutations of 1,2,...,2N for which i) < ji, i2 < jo, ..., in < jn and 
ij <i2 <+++<iy; such permutations are in one-to-one correspondence with the ways 
of dividing 1,2,...,2N into pairs. Kasteleyn showed that if a graph G is planar, then 
there is an antisymmetric matrix A whose /, jth entry is a unit complex number when 
the ith and jth vertices of G are adjacent and 0 otherwise, such that the modulus of 
the Pfaffian of A equals the number of perfect matchings of G. Moreover, the deter- 
minant of A equals the square of the Pfaffian of A, and hence has modulus equal to 
the square of the number of perfect matchings of G. This is the Hafnian-Pfaffian 
method (where loosely speaking the Hafnian is to the Pfaffian as the permanent is to 
the determinant; see [101] for details). 

Kasteleyn’s method also works for graphs with (positive) edge-weights attached; 
one simply multiplies the weights in the weighted adjacency matrix by the required 
“Kasteleyn phases.” Kasteleyn showed how to modify the method to cope with 
graphs on surfaces of small genus, but a discussion of this would take us too far 
afield. 
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We return to the bipartite case. It is known that the entries of the inverse 
Kasteleyn-Percus matrix K~! have significance of their own; for instance, when the 
ith white vertex and the jth black vertex are adjacent, the absolute value of the j, ith 
entry of K~! is the probability that the edge joining those two vertices will belong 
to a matching chosen uniformly at random. (For this and much else on probabilistic 
aspects of matchings, see [83].) A recent treatment of the inverse Kasteleyn matrix 
for Aztec diamond graphs is [22]. 

Kuperberg [105] has shown that the Gessel-Viennot matrix for a bipartite pla- 
nar graph (typically of size O(n)) and the Kasteleyn-Percus matrix of that graph 
(typically of size O(n7)) are not really different entities, and that it is possible to 
turn the latter into the former by means of combinatorially meaningful determinant- 
preserving matrix reductions. Extending this to the Pfaffian case for non-bipartite 
matching remains an open problem. 


9.3.3 The spanning tree method 


Figure 9.13 shows a domino tiling of a 5-by-5 square with a corner cell removed. The 
centers of certain cells have been marked, displaying a 3-by-3 grid within the 5-by-5 
grid; when a domino contains a marked square, we draw an arrow from the center 
of its marked square to the center of its unmarked square. The dominos containing a 
marked square determine a spanning tree rooted at the missing corner, as shown in 


UI lee 

—= T 

T == 
Figure 9.13 


A tiling and a tree. 


Temperley [164] gave a method for turning certain dimer problems into spanning 
tree problems. This method was extended [86] and later was shown to apply to all 
bipartite planar graphs [87]. 
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Figure 9.14 
The tree made clearer. 


The strength of the method hinges on the fact that the Matrix Tree theorem (see 
for example Theorem 5.6.8 of [158] or p. 57 of [10]) gives a formula for the number 
of spanning trees of a graph as the determinant of an associated matrix. For applica- 
tions, see [29, 31, 117, 118]. To my knowledge, no one has as of yet exposed a link 
between the Kirchhoff matrix of a bipartite planar graph and either the Kasteleyn- 
Percus matrix or the Gessel-Viennot matrix. 

The connection between tilings and trees yields a connection with the abelian 
sandpile model; see [60]. 

Matchings and spanning trees fit into a bigger picture involving specializations 
of the Tutte polynomial. See [169] as well as [154] and [21]. 


9.4 Representation-theoretic methods 


(Note: Much of the information in this section was kindly provided by Greg Kuper- 
berg.) 

There are two broad classes of applications of representation theory to enumer- 
ation of matchings, which we might call “trace” and “determinant” methods. In the 
trace method, one interprets the number of tilings as the trace of some matrix and 
then uses representation theory to compute that trace. (In many cases, the matrix in 
question is the identity matrix so the trace is just the dimension of the vector space.) 
In the determinant method, one interprets the number of tilings as the determinant of 
some matrix and then uses representation theory to compute that determinant. 

Larsen’s proof of the formula for the number of domino tilings of an Aztec dia- 
mond [53] is an example of the trace method. Kuperberg [100] and Stembridge [160] 
(the latter building upon Proctor [132]) independently applied this approach to the 
study of lozenge tilings of hexagons under various symmetry constraints. In some of 
these results, the matrix one studies is the permutation matrix of an involution, so 
that the trace counts the fixed points of the involution. 
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Examples of the determinant method can be found in [103]. Here, one uses the 
representation theory of the Lie algebra sl(2). If e, f, and h form a basis for sl(2) in 
the usual way, a block of the ladder operator e goes from the h = —1 eigenspace to 
the = +1 eigenspace, and the number of tilings equals the determinant of this block 
(up to a normalization factor). The representation theory of sl(2) lets one diagonalize 
this operator and thereby compute the determinant. This works for various symmetry 
classes of lozenge tilings of hexagons. It would be interesting to see trace methods 
applied to symmetry classes of domino tilings of Aztec diamonds. 

Representation-theoretic arguments have also been successful in enumerative 
problems involving ASMs (Alternating Sign Matrices). Although the initial view 
of ASMs in [122] does not seem like it has anything to do with tilings, there is 
an attractive interpretation of ASMs as tilings that fill regions called “supergaskets” 
with curved tiles called “gaskets” and “baskets.” See Figure 9.15 for a picture of 
such a tiling, and the article [138] for an explanation of the link between tilings 
of this kind and ASMs viewed as n-by-n arrays of +1’s, —1’s, and 0’s satisfying 
various constraints. It is also worth mentioning that domino tilings of Aztec dia- 
monds were in a sense discovered by Robbins and Rumsey [149] as an outgrowth 
of their work on ASMs, even though the tilings picture was not made explicit. The 
picture of ASMs as tilings has not been helpful to researchers, but the dual picture 
of ASMs as states of the six-vertex model (see Figure 9.16) plays a key role in Ku- 
perberg’s proof [102] that the number of ASMs of order n is equal to the product 
[jz (3k + 1)!/(n +4)! (conjectured by Mills, Robbins and Rumsey [122] and first 
proved by Zeilberger [176]). 

In the six-vertex model (also called the square ice model), edges of a square 
grid are assigned orientations in such a way that at each vertex there are two arrows 
pointing in and two arrows pointing out. (At each vertex, there are (3) = 6 possible 
ways to choose orientations for the incident edges, hence the name of the model.) 
As in the case of domino and lozenge tilings, we find that the boundary conditions 
for the six-vertex model that are most relevant to physics are not the ones most con- 
ducive to exact enumeration. Just as the logarithm of the number of domino tilings 
of the Aztec diamond falls short of the full entropy of the dimer model on the square 
grid, the logarithm of the number of alternating-sign matrices falls short of the full 
entropy of the square ice model. Curiously, although the former entropy is not be- 
lieved to be the logarithm of an algebraic number, the latter number is the logarithm 
of the algebraic number 8/(3\/3) (traditionally written as (4/3)>/? in the physics lit- 
erature, starting with its original appearance in [112]). Although the model with free 
boundary conditions is exactly solvable in the sense of physics (that is, in the sense 
that one can find a formula for the entropy), it is not known to be exactly enumerable 
(in the non-asymptotic sense). For exact enumerations, it appears one must impose 
the “domain-wall boundary conditions” (so dubbed by Korepin; see [91]) that pre- 
vail in the particular square ice configurations that correspond to ASMs. We will use 
this term in a more general way to denote boundary conditions in which different 
“frozen” phases of the system govern the behavior of different parts of the boundary, 
so that the interior is subject to competing influences (and can spontaneously divide 
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Figure 9.15 
A tiling by baskets and gaskets (one of 42), with the associated alternating-sign ma- 
trix. 


itself into multiple macroscopic domains in which the tiling displays qualitatively 
different microscopic behavior). 

For enumeration of ASMs of order n, the relationship with representation the- 
ory has been quite intimate, and some of the proofs in the literature make uses of 
different sorts of representation theory arguments at different stages in the proof. In 
the first place, when one relaxes the grid-structure underlying the square ice model 
and looks at ice-configurations of a more general kind, one finds Reidemeister-style 
moves that preserve the partition function or affect it in predictable ways; this is the 
theory of Yang-Baxter relations, and it is tied in with the g-deformed representation 
theory of sl(2). Against this background, Izergin and Korepin were able to write the 
partition function in terms of the determinant of an n-by-n matrix [73, 91] involving 
2n variables associated with the rows and columns of the ice-grid. When one seeks 
to apply this formula to count ASMs, one obtains an indeterminate of type 0/0, but 
by approaching the singularity along an artfully chosen path, Kuperberg was able to 
obtain the desired enumeration. For more details, see [102] and [106]. Okada [127] 
brought Lie group characters into the picture, with two distinct quantum parame- 
ters playing roles (one tending toward 1 as in Kuperberg’s original proof, the other 
a cube root of unity). Okada showed that for all the proved or conjectural formulas 
in the second of Kuperberg’s articles, the requisite determinant evaluations are as- 
sociated with characters of Lie groups; for instance, the formula Kuperberg proved 
in [102] is associated with an irreducible representation of GL(2n). Symmetry classes 
of alternating-sign matrices have also been enumerated by Razumov and Stroganov 
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Figure 9.16 
A state of the six-vertex model with domain-wall boundary conditions. 


[144, 145, 146]. As in the case of other tiling problems with nice answers, we find 
that enumeration with the extra constraint of symmetry also gives rise to tiling prob- 
lems with nice answers (which are usually harder to prove). 

A central mystery of the subject is the remarkable fact that the number of fully 
symmetrical lozenge tilings of a regular hexagon of side 2n (which are in bijec- 
tion with Totally Symmetric Self-Complementary Plane Partitions of order 2n) are 
equinumerous with ASMs of order n. Both are governed by the product formula that 
appears above, but there is not at this time a conceptual explanation of why the two 
numbers should be the same. For instance, despite intensive work by combinatori- 
alists over the past several decades, no bijection between TSSCCPs and ASMs is 
known. 


9.5 Other combinatorial methods 


A very fertile approach to enumeration of matchings, discovered by Eric Kuo while 
he was an undergraduate member of the Tilings Research Group at MIT, is a graphi- 
cal analogue of Dodgson condensation. (For background on Dodgson condensation, 
see [16].) Figure 9.17 shows an Aztec diamond graph G of order 4 divided by two 
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Figure 9.17 
Kuo condensation for Aztec diamonds. 


diagonals of slope +1 and two diagonals of slope —1. If one cuts the figure with 
one of the two lines of slope +1 and one of the two lines of slope —1, one obtains an 
Aztec diamond graph of order 3 (and three smaller regions). Let Gy, Gs, Gg, and Gw 
respectively be the Northern, Southern, Eastern, and Western 3-by-3 Aztec diamond 
subgraphs obtained in this way, and let Gc be the central 2-by-2 Aztec diamond sub- 
graph. Then the original version of Kuo condensation [99, Theorem 2.1] tells us 
that M(G)M(Gc) = M(Gy)M(Gs) + M(Ge)M(Gy), where M(-) denote the num- 
ber of matchings of a graph. In this case, since the graphs Gy, Gs, Gg, and Gw are 
isomorphic, we obtain the relation M(G4)M(G2) = 2M(G3)* (where G, denote the 
Aztec diamond graph of order n), and more generally M(Gy4.1)M(Gn_1) =2M(G,)?, 
which allows us to solve for M(G,) using the initial conditions M(Go) = 1 and 
M(G}1) = 2. See [99] and [98]. For more details on Kuo condensation and numer- 
ous applications, see (in chronological order) [172, 171, 90, 62, 35, 36, 34, 23]. 

When one applies Kuo condensation repeatedly, keeping track of all the differ- 
ent subgraphs of the original graph that occur, one accumulates data that are mostly 
easily stored in a three-dimensional array in which the entries form an octahedral lat- 
tice (dual to a cubical lattice), in which the numbers M(G), M(Gc), M(Gy), M(Gs), 
M(Gz_), and M(Gy) are associated with vertices that form an octahedron. That is 
to say, the enumerations of matchings satisfy the octahedron relation. This equation 
originated in the theory of discrete integrable systems, and has found numerous ap- 
plications in combinatorics. Here it is worth pointing out that Kuo condensation can 
be applied with weightings of a fully general kind, where each edge is assigned its 
own weight independently of all the others. 

A superficially different but probably related approach to enumerating matchings 
is (generalized) domino-shuffling [140], also known as urban renewal. There is 
some confusion in the literature regarding the intended meanings of these two terms. 
At least originally, urban renewal referred to a certain kind of local modification of 
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(a) (b) 


Figure 9.18 
Urban renewal. 


an edge-weighted bipartite planar graph that has a predictable effect on the partition 
function. In this it drew much inspiration from the Temperley-Fisher method [165] of 
enumerating matchings, which used similar local mutations to turn states of the dimer 
model into states of an Ising model on a suitable graph. (See Chapter 13 of [125], 
particularly Sections 13.6 and 13.7. Also see [121], in which the the dimer and Ising 
models are solved by use of Pfaffians in quite complete detail. Yet another worth- 
while article on the subject is [123].) The inspiration for the terminology drew upon 
an image of small pieces of the graph as being “cities” that communicate with the 
outside world via certain ports (as well as being an obscure pun on a style of urban 
planning that had a brief vogue in the 1970s). Strictly speaking, urban renewal is an 
operation not on graphs but on edge-weighted graphs. By performing such operations 
throughout a graph in a systematic fashion (this is what was meant by “generalized 
domino-shuffling,” on account of its relationship to the domino-shuffling algorithm 
introduced in [53]), one can often obtain a simpler graph of the same kind, giving 
rise to a recurrence relation for the number of matchings. In addition to having “local 
factors” associated with individual edges, it is often handy to collect together “global 
factors” that are associated not with individual edges but with the graph as a whole. 
The basic urban renewal operation is depicted in Figure 9.18 If an edge-weighted 
graph G having a city of the kind shown in panel (a) is modified by replacing that 
city by a city of the kind shown in panel (b) (remaining otherwise unchanged), with 
w = 2/(wz+xy), x = y/(wzt+xy), ¥ = x/(wz+xy), and z! = w/(wz+xy), then 
the sum of the weights of the matchings of the first graph equals wz + xy times the 
sum of the weights of the matchings of the second graph. Like Kuo condensation, 
generalized domino-shuffling can be applied with weightings of a fully general kind, 
where each edge is assigned its own weight independently of all the others. Using 
urban renewal allows one to show fairly directly that the number of matchings of 
the Aztec diamond graph of order n equals 2” times the number of matchings of the 
Aztec diamond of order n — 1. Urban renewal was used in [173] to show that, for a 
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family of finite graphs derived from the infinite square-octagon graph, the number of 
matchings is given by 5” or 2x 5™ (depending on which member in the family is 
being discussed). 

Di Francesco [50] gives a way to think about generalized domino-shuffling that 
unifies it with the octahedron recurrence, via certain sorts of discrete surfaces in the 
octahedron lattice. 

The usefulness of generalized domino-shuffling is partly limited by the adher- 
ence to a square grid. Ciucu’s complementation method [26] remedies this defect 
by introducing the idea of one weighted graph H’ being the complement of another 
weighted graph H relative to some ambient graph G (the “cellular completion” of 
#1), and establishing a formula relating the partition functions for matchings of H 
and matchings of H’ [26, Theorem 2.1]; the A-factors in that theorem can be recog- 
nized as a generalization of the face-factors in generalized domino-shuffling. A sim- 
ilarly flexible factorization theorem [25] gives a way to write the partition function 
of a weighted plane graph with reflection-symmetry as the product of the partition 
functions of two smaller graphs (up to a predictable multiplicative factor). For appli- 
cations, see [27] and [109]. It is worth mentioning that Yan and Zhang [172] used 
Ciucu factorization to prove a version of Kuo condensation. Thus in a sense Ciucu 
factorization implies Kuo condensation. 

Finally, a different sort of proof of the enumeration of lozenge-tilings of a 
hexagon comes from Krattenthaler’s bijective proof [94] of Stanley’s hook-content 
formula for the generating function for semistandard Young tableaux of a given 
shape. His proof yields, as a spin-off, a way of sampling from the uniform distri- 
bution on the set of lozenge tilings of a semiregular hexagon (by way of associated 
semistandard tableaux). It is perhaps too isolated an achievement to be considered a 
method, but it deserves study and imitation. 


9.6 Related topics, and an attempt at history 


The monomer-dimer model was first considered as a simplified model of a gas [61] 
and later as a model of adsorption of diatomic molecules on a surface. However, 
the mathematical difficulties posed by the monomer-dimer model in two or three 
dimensions, combined with a certain amount of mismatch between physical reality 
and the simplified model, did not lead to successful applications of the theory. 
Fortunately for combinatorialists, physicists like Fisher, Kasteleyn and Temper- 
ley were not dissuaded and, emboldened by Onsanger’s success [128] in exactly 
solving the two-dimensional Ising model, invested considerable effort and ingenu- 
ity in finding exact solution methods (in Sections 9.2—9.5) for the pure dimer model 
[78, 165], even though that model is even more removed from the aforementioned tar- 
get applications than the monomer-dimer model. Lieb’s exact solution of the square 
ice model [1 12]—that is, his calculation of the per-site entropy—and the close agree- 
ment between Lieb’s answer and the experimentally determined residual entropy of 
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actual (non-square!) three-dimensional ice gave further impetus to the study of two- 
dimensional lattice models in statistical mechanics. 

The statistical mechanics literature (to which [5] serves as an excellent introduc- 
tion) provides a unifying point of view for models that at a combinatorial level might 
seem to be unrelated. For instance, the dimer model on a square grid and the square 
ice model can both be seen as special cases of the eight-vertex model (an extension 
of the six-vertex model that permits vertices with all arrows pointing in or all arrows 
pointing out), with different sorts of weights attached to the edges of the graph; the 
dimer model is easier to solve than the square ice model because the former is part 
of the “free-fermion regime” of the eight-vertex model [56]. Statistical physicists 
Grensing, Carlsen, and Zapp [68] were the first researchers in any subject to look 
at domino tilings of Aztec diamonds, or rather (in their way of looking at things) 
dimer configurations in the dual graph. They went so far as to assert without proof 
the formula for the number of dimer configurations, but their published explorations 
stopped there, presumably because the model did not give rise to the known per-site 
entropy; they thereby missed out on discovering not just a beautiful combinatorial 
theorem but a rich body of theory that grew out of it, relating the dimer model to 
PDE and algebraic geometry [83]. This story bears out the observation that undis- 
covered mathematical treasures often lie in the trashbins of physicists. 

A more harmonious match between theory and reality arose from work of theo- 
retical chemists studying benzenoid hydrocarbons. These molecules, often depicted 
as having rings of alternating single and double bonds, actually have a more subtle 
quantum-mechanical nature, and can be viewed to a first approximation as being in a 
superposition of different valence-states or “Kekulé structures” [67]. The number of 
Kekulé structures on a graph has implications for its chemical properties, so chemists 
developed methods for counting these structures; indeed, a version of the Lindstr6m- 
Gessel-Viennot method discussed above was found independently by Horst Sachs 
and his co-workers. (The class of regions mentioned in Example 9.1.2 of Section 9.1 
was originally studied by chemists.) See [77], [47], and [151]. 

For a more recent application of tilings to the physical sciences, see [9], which 
like [61] uses the monomer-dimer model as a model of adsorption of diatomic 
molecules on a surface. 

There has also been work on stepped surfaces (e.g., [8], [110], and [59]) of which 
tilings can be seen as two-dimensional projections (cf. our earlier discussion of Fig- 
ure 9.6 and its interpretation as the three-dimensional Young diagram of a plane 
partition). Here the interests of physicists have a certain amount of overlap with the 
interests of probabilists and representation theorists. A phrase that has recently been 
associated with this line of work is “integrable probability theory” [11]. 

Probabilists are often interested in limit-shapes arising from tiling models when 
one takes the uniform distribution (or, more generally, distributions arising from a 
weighting) on the set of tilings of a large region and sends the size to infinity. For both 
random domino tilings of large Aztec diamonds and random lozenge tilings of large 
regular hexagons, one sees (with overwhelmingly high probability) a sharp boundary 
between a central circular region in which tiles of different orientations are mixed to- 
gether and outer regions in which all tiles are aligned with their neighbors [39, 41]. 
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There is a great deal of interest in the corresponding problem for alternating-sign ma- 
trices; there is a widely-believed conjecture for the asymptotic shape of the bound- 
ary between the central “liquid” domain and the four outer “solid” domains [83] 
of large random ASMs, but no rigorous proof is currently known. See [42], [43], 
and [44]. 

Probabilists are also interested in algorithms for randomly sampling from the uni- 
form distribution (or, more generally, distributions arising from a weighting) on the 
set of tilings of a large region. In the case of matchings of planar graphs, a number of 
methods are available; see Section 9.8 for a discussion of this. Note that one method 
for random sampling is to iteratively make decisions in accordance with conditional 
probabilities (obtainable as ratios of tiling-enumerations); in the case of matchings 
of planar graphs, this is feasible because of Kasteleyn’s method, and some tricks 
introduced by David Wilson [170] make this competitive with other methods. This 
method of exactly sampling from the uniform distribution cannot currently be ap- 
plied to alternating-sign matrices because we lack exact formulas for the relevant 
conditional probabilities. It seems unlikely that the counting problems that arise are 
anywhere close to #P-complete, but we cannot currently rule out this possibility. 

The broad applicability of Kasteleyn’s method has inspired a whole line of re- 
search in computer science called the theory of holographic algorithms, in which 
one tries to see how many other counting problems one can solve using vari- 
ants of Kasteleyn’s trick, via computational elements called matchgates. See [70] 
and [18]. 

The Aztec diamond and its relatives have also played a role in the study of 
random matrices, the connecting link being fluctuations in the boundaries between 
phase-domains; e.g., in the Aztec diamond of order n, fluctuations are of order ni/3 
and converge in distribution to the Tracy-Widom distribution. For more about bound- 
ary fluctuations, see [76]. For an account more focused on lozenge tilings, see [58]. 

Kuo’s condensation lemma is related to the octahedron relation (also called the 
discrete Hirota equation; see Section 12 of [97]). In this way, the study of enumera- 
tion of matchings leads to the theory of (discrete) integrable systems, and has con- 
nections to cluster algebras; [66] and [50] are good sources of more information. 

Lastly, it should be mentioned that tilings, like other combinatorial objects, often 
arise in connection with algebraic objects. Standard examples of this phenomenon 
are matchings of complete bipartite graphs (which are implicit in the formula for the 
determinant of a matrix expanded as a sum of n! monomials) and Young tableaux 
(which occur throughout the representation theory of the symmetric group). Match- 
ings of Aztec diamond graphs, in disguised form, index the terms the expansion of 
the A-determinant of Robbins and Rumsey [149] as a sum of Laurent monomials. 
Since lozenge tilings of a semi-regular hexagon can be converted to semistandard 
Young tableaux, they index terms in Schur polynomials associated with rectangular 
Young diagrams. More recently, square-triangle tilings of equilateral triangles have 
been used in a modern reformulation of the Littlewood-Richardson rule for express- 
ing the product of two Schur functions as a linear combination of Schur functions 
[143]; see [177]. 
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9.7 Some emergent themes 
9.7.1 Recurrence relations 


When we have a sequence of tiling problems, the answer is the integer sequence 
a\,a2,... whose nth term counts the number of solutions to the nth tiling problem. 
What kinds of questions have nice answers? Niceness is subjective, so for present 
purposes we define a “nice answer” to be a sequence that appears in the mathematical 
literature and can be found via Sloane’s On-line Encyclopedia of Integer Sequences 
(http: //oeis. org) or has patterns that can be inferred empirically (e.g., satisfies a 
linear recurrence relation). When a, grows exponential in n? (as is usually the case 
for interesting two-dimensional tiling enumerations), there is no chance of finding a 
linear recurrence. So what else can we try? 

One answer is, rational recurrences. The sequence ad, = gnin+l)/2 satisfies the 
recurrence dyj41 = 2a /an—1, and this is not accidental; writing the recurrence in 
the form dy—1dn41 = AnGn + Andy, We see it as an instance of Kuo condensation. 
Applying this insight in reverse, one can start with a nonlinear recurrence like 
An—24n42 = Ay—-1 An+1 +a (satisfied by the Somos-4 sequence 1,1,1,1,2,3,7,23,59.... 
described in [64] and [63]) and reverse-engineer an appropriate family of bipartite 
planar graphs whose nth member has a, matchings; see [14] and [155]. Just as the 
existence of a linear recurrence satisfied by the sequence a},a2,... is signaled by the 
existence of a value of m for which the Hankel matrix 


ant+1 an+2 see an+m 

Qn+2 an+3 sss Ant+m+1 
A (dn41,4n42; ee An+2m—1) = 

Qnt+m 4ntmtl +++ 4n+2m-1 


appears to be singular for all n in some range, the existence of a Somos-type recur- 
rence satisfied by the sequence a 1,a2,... is signaled by the existence of values of k 
and m for which the Somos matrix 


A (an41,--+,Gn42m—1) * T (An4i 15+++54n+k+2m 1) 


appears to be singular for all n in some range, where T(...) denotes the Toeplitz ma- 
trix obtained by writing each row of H(...) backwards and * denotes the Kronecker 
product of matrices (Somos, personal communication). For example, the 3-by-3 So- 
mos matrix 


Qa\a5 a2a4 a3a3 
H (a1, 42,43,44,a5) *T (a3,44,45,06,47) = | 246 a3a5 a4a4 
Qa3AQ7 a4dg a5da5 


is singular when aj,a2,... satisfies the Somos-4 recurrence, and this remains true 
when all subscripts are shifted by adding a constant. It seems possible that some 


566 Handbook of Enumerative Combinatorics 


tiling-enumeration problems for which no exact formula has been discovered will 
turn out to governed by nonlinear recurrences, which might be discoverable by use 
of Somos matrices and related tools. 

Inasmuch as rational recurrences like Somos-4 often arise as projections of a 
three-dimensional octahedron relation, an attractive research strategy for finding 
a nonlinear recurrence might be to imbed the sequence as a strand of a higher- 
dimensional array that satisfies the octahedron relation. This is one way to think 
about the approach to counting domino tilings of squares used in [141], although of 
course that construction had the benefit of hindsight (in the form of Kuo condensa- 
tion). 


9.7.2 Smoothness 


In the absence of visible patterns linking the terms of a sequence, one sign of an 
underlying regularity is properties of the individual terms themselves, as integers. 
One sort of pattern can appear when one looks at how the integers factor into 
primes. Call a sequence a1,a2,..., smooth if the largest prime factor of a, is O(n), 
and ultra-smooth if the largest prime factor of a, is O(1), that is, bounded. (An- 
other word that is sometimes used instead of “smooth” is “round.”) For example, 
H(3n)H (n)? /H(2n)? (the number of lozenge tilings of a regular hexagon of side 1) 
is smooth; gn(n+1)/2 (the number of domino tilings of an Aztec diamond of order n) 
is ultra-smooth. Quite often we find that the answers to enumerative questions are 
smooth sequences, and the smoothness is often a key to finding an exact product 
formula. 

Powers of 2 arise from looking at matchings of Aztec diamond graphs [53] asso- 
ciated with the regular tiling of the plane by squares; powers of 5 arise from looking 
at matchings of “fortress graphs” [173] associated with the semiregular tessellation 
of the plane by squares and octagons; and powers of 13 arise from looking at match- 
ings of “dungeon” and “Aztec dungeon” graphs [27] associated with the semiregular 
tessellation of the plane by squares, hexagons, and dodecagons, respectively. In all 
three cases, there is a highly symmetrical tessellation of the plane in the background, 
and the tiling problem is best thought of as the problem of counting matchings of 
certain finite subgraphs of the highly symmetrical infinite bipartite planar graph that 
arises as the dual of the tessellation. We can sharpen our inquiry by asking: what 
sorts of tessellations, in combination with what sorts of subgraphs of the graph dual 
to the tessellation, give rise to “ultra-smooth enumerations,” i.e., enumerations in 
which only finitely many prime factors arise? 

Regarding the choice of tessellation, it was observed by David Wilson that if you 
deform the square-hexagon-dodecagon graph so that it becomes a subgraph of the 
square grid, turn that into a square grid with edge-weights 0 and 1, and then per- 
form generalized domino-shuffling (hereafter, “shuffling”) three times, you get back 
the original edge-weights (modulo translation). This is analogous to Yang’s discov- 
ery [173] that if you use urban renewal to turn the square-octagon graph into a square 
grid with edge-weights and apply shuffling twice, you get back what you started with 
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(thereby obtaining exact enumeration of “diabolo tilings of fortresses’), and also 
analogous to the original application of urban renewal to Aztec diamonds [136]. 

More generally, consider an infinite (but locally finite) bipartite planar graph G 
with two independent translation-symmetries. Suppose that it gives rise, via a suitable 
sort of deformation, to a doubly periodic weighting of the infinite square grid with 0’s 
and 1’s (with a caveat discussed below). Call this weighting Wo. Applying shuffling to 
Wo, one gets some weighting W,; applying shuffling again, one gets some weighting 
W.; etc. Suppose that W,, = Wo for some m. Each W; is doubly periodic, and hence 
has only finitely many distinct “face-factors,’ where the face-factor associated with 
a face is ac-+bd where a,b,c,d are the consecutive edge-weights around the face. 
Hence only finitely many primes occur in the numerators of face-factors arising from 
the weightings Wo, Wi, ..., Win—1. These are the primes that occur in ultra-smooth 
enumerations of matchings of finite subgraphs of G. 

Therefore, one question that is likely to shed light on the subject of ultra-smooth 
enumeration is, what doubly periodic graphs G give rise to weightings of the square 
grid that come back to themselves after a finite number of iterations of shuffling? 
This is probably not the most natural setting for the question, since it gives the square 
grid an unjustified privileged role, and not all ultra-smooth enumerations arise in this 
way, but it is a place to start. (For more on iterated shuffling as an integrable system, 
see [66].) A related project is to classify triply periodic solutions to the octahedron 
recurrence (with periodicity in two “space directions” and one “time direction’). 

However, there is more to the story than the infinite graph G: One must also 
pick out the correct sorts of finite subgraphs. Here we face an irony that drastically 
augments the disappointment Grensing, Carlsen, and Zapp must have felt in their 
abortive study of Aztec diamond graphs: The sorts of subgraphs that are amenable to 
simple combinatorial methods never include the sorts that are relevant to computing 
the per-site entropy of the dimer model on the infinite graph. If for instance we ap- 
ply shuffling to compute the number of domino tilings of a 2n-by-2n square, in the 
interior we pick up the face-factors equal to 2, but along the boundary we find other 
numbers that increase in complexity as we apply new rounds of shuffling, and we 
ultimately lose control over the analytical form of the answer. 

In “designing” various families of bipartite planar graphs that turned out to have 
ultra-smooth enumerations, a heuristic that has proved to be very successful, but 
whose success is quite mysterious, is what one might (borrowing physicists’ nomen- 
clature) call the domain-wall heuristic. This is illustrated by the considerations that 
led me to (re-)invent Aztec diamonds in the 1980s. I was aware of MacMahon’s 
beautiful product formula for the (smooth, though not ultra-smooth) enumeration of 
lozenge tilings of hexagons, and Kasteleyn’s formula for the (un-smooth) enumer- 
ation of domino tilings of squares, and wondered what was responsible for the dif- 
ference. A reading of W. Thurston’s article [166], focusing on the concept of height- 
functions for these two sorts of tilings, led me to notice that the height-function along 
the boundary of the square is (modulo tiny fluctuations) flat, while the height func- 
tion along the boundary of the hexagon has dramatic rises and dips. Might a region 
in the square grid whose boundary consisted of four zigzag paths, with dramatic rises 
and dips in the height function, be a better analogue of the hexagon, and have similar 
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nice enumerative properties? The answer was a resounding “yes”; nothing is more 
ultrasmooth than a sequence of powers of 2. 

Similar considerations, applied to other doubly periodic graphs, yielded other 
conjectural smooth enumerations, such as the ones later proved by Yang and by 
Ciucu. In every case, the starting point was a two-colored doubly periodic tessel- 
lation (often obtained from [69]) in which black cells are surrounded by white cells 
and vice versa. In such a tessellation one draws infinite paths that abut only white 
cells on one side and only black cells on the other side. A collection of such zigzags 
divides the plane into subregions. The finite regions that arise in this fashion turn out 
suspiciously often to be associated with ultra-smooth enumerations. 

With these sorts of domain-walls boundaries, it is inevitable the tiling will be 
more tightly constrained near the boundary than in the interior, and so it is not 
surprising that segregation of the region into different phase-domains with interest- 
ing asymptotic shapes can occur. However, it still seems magical that domain-wall 
boundaries give rise to ultra-smooth enumerations. 

Moreover, in the case of lozenge tilings of hexagons, the explanation in terms of 
face-factors does not apply, and we do not get an ultra-smooth enumeration, but we 
still get a smooth one, with prime factors that grow only linearly in size. The same 
observation applies to gaskets-and-baskets tilings: when one chooses paths through 
the plane that lift to maximally steep paths in the height-function picture, the regions 
the paths cut out are supergaskets, whose tilings (which are in bijection with ASMs) 
yield smooth enumerations. 

Now comes the caveat that I warned you about: Kuperberg’s method of turning 
general bipartite planar graphs into subgraphs of the square grid gives rise to weight- 
ings with lots of 0’s, and these are not always amenable to shuffling when expressions 
of the form 0/0 arise. The article [74] discusses some ways of circumventing this 
problem, but a complete solution awaits discovery. This issue has connections with 
a similar issue that arises when one carries out Dodgson condensation to evaluate 
determinants of matrices [16]. 


9.7.3 Non-periodic weights 


With q a formal indeterminate, define the g-integer [n] = 1 +q+---+q""!, the 
q-factorial [n]! = [1][2]---[n], and the g-hyperfactorial H({n]) = [1]![2]!---[2—1]!. 
Many of the integer-valued enumerations associated with matchings of graphs can 
be obtained as the g = | specializations of more refined g-enumerations, in which 
one “q-counts” tilings by summing the quantity qi?) as T varies over all the tilings, 
where /1 is some integer-valued “global height” associated with T. More specifically, 
one can write (T) as a sum of local heights associated with the vertices of the tiling 
(or with the faces of the dual graph), as in W. Thurston’s exposition [166] of Con- 
way’s approach to tilings [45]; see also [134]. 

One can fit g-enumeration into the framework of edge-weighted enumeration, but 
at a cost: the pattern of weights will no longer be periodic. For instance, looking at a 
regular procession of edges, one may find that their edge weights follow the pattern 


Cae Magy we Gee 
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Factorizations in Z|q] are more informative than factorizations in Z; 12 could be 
2 times 6 or 3 times 4, but [2] times [6] is different from [3] times [4], so in the case 
of smooth enumerations that give product formulas, empirical g-enumerations for 
small regions give more clues about what the general formula might be. In this way 
q-analogues can sometimes provide information that illuminates the g = 1 enumer- 
ations. A recent example of this (applied to a nonsmooth enumeration) came from 
Adam Kalman’s work on Aztec pillows, discussed in the 2015 version of [137]. In- 
deed, I like to think of the step of replacing Z by Z[q] as just the first step down a 
road that, if you can reach the end, replaces what started out as a mere number by 
a multivariate polynomial (or Laurent polynomial) in which every coefficient is 1. 
In this way, the monomials themselves are seen to encode the combinatorial objects 
being counted. This “very many variables” point of view, which draws its inspiration 
from its success in the case of Aztec diamonds, was a key ingredient in the work of 
Carroll and Speyer on groves [20]. 

It should be mentioned (apropos of Kalman’s work) that there really is not a 
hard dichotomy between smooth and nonsmooth enumerations; in particular, a single 
enumeration can exhibit both smooth and nonsmooth features, with certain small 
primes occurring with very high exponents and larger primes occurring with much 
smaller exponents. 

A recent and interesting way to add variables, different from the height-functions 
approach, is the “elliptic” approach of Borodin, Gorin, and Rains [12]. 


9.7.4 Other numerical patterns 


Sometimes in examining the prime factorization of an enumeration, one sees domi- 
nance not of a particular finite set of primes but of an infinite arithmetic progression 
of primes. This is the case for the sequence whose terms count domino tilings of 
squares: one sees numbers in whose prime factorizations primes congruent to | mod 
4 predominate. 

Sometimes the patterns one sees involve not the primes but their exponents. For 
instance, when all the exponents in an enumeration are even (that is, when the answer 
is a perfect square for all n), surely there is something going on that demands an 
explanation. A variant of this phenomenon is that certain answers to tilings problems 
are always twice a perfect square. Typically such patterns are explained by appealing 
to symmetries of the graph (e.g., [75] and [25]). 

Sometimes one sees patterns in the congruence classes of the enumerations 
themselves (not their prime factors). The prototype for this behavior is seen in 
the Fibonacci numbers, which are periodic mod m for every m, and more gener- 
ally any sequence whose terms satisfy a linear recurrence relation over the inte- 
gers whose characteristic polynomial has first and last coefficients equal to +1. It 
is less obvious that this periodicity property should be true for the Somos-4 sequence 
1,1,1,1,2,3,7,23,59,... (since the defining recurrence is not linear but rational) but pe- 
riodicity still holds in this case [120] and in other similar cases. In the case of tiling 
problems, Cohn has shown that the sequence whose nth term counts domino tilings 
of the 2n-by-2n square is periodic modulo m whenever m is a power of 2 [38]. 
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Divisibility properties constitute a special case of congruence properties. For one 
example of a result asserting divisibility, see [72], which shows that the number of 
perfect matchings of any member of a certain family of graphs is always divisible 
by 3. 

Lastly, one sometimes sees that later terms of a sequence tend to be divisible by 
certain earlier terms. In the case of the Fibonacci sequence (which counts domino 
tilings of rectangles of width 2), this observation goes back centuries. A general- 
ization of this classical fact that applies to domino tilings of rectangles of arbitrary 
fixed width was proved by Strehl [162] (see also [137, Problem 21]). Quite recently, 
MIT undergraduate Forest Tong has found a combinatorial explanation for a broad 
generalization of this phenomenon [167]. 


9.7.55 Symmetry 


When a region has symmetries, one can count tiling of the region that are invariant 
under the full symmetry group of the region or some subgroup. Choosing a larger 
subgroup will of course reduce the number of tilings, but curiously, even though the 
numbers get smaller, the problems get harder! (Though sometimes when the sub- 
group is too large, the number of tilings drops to 0, and then the problem is trivial.) 
The original example of this sort of story is Stanley’s article on symmetry classes of 
plane partitions [156]; the smaller subgroups were the first ones to be settled (starting 
with the trivial subgroup, which was settled by MacMahon), culminating in the case 
of Totally Symmetric Self-Complementary Plane Partitions (TSSCPPs). The final 
piece of this ten-symmetry-classes-of-plane-partitions story was not put into place 
until 2011 [92]. 

Matchings that are invariant under a symmetry can be thought of as matchings of 
a quotient graph, although some care is required when the action is not free: Vertices 
or edges whose orbit is of anomalously small cardinality require special treatment in 
“orbigraph matchings” (see [104]). 

More mysterious than the above is the g = —1 phenomenon discovered by Stem- 
bridge and significantly generalized by Reiner, Stanton, and White in their notion of 
the cyclic sieving phenomenon [147] (see also [152] and [148]). Consider the set S$ 
of lozenge tilings of some hexagon, either unconstrained or constrained to possess 
certain symmetries of the hexagon. Let f(g) be the g-enumeration of the tilings. It 
is clear that f(1) is the number of tilings in the set S we are looking at. What is 
surprising is that, in certain cases of this kind, f(—1) turns out to have enumerative 
significance of its own: It is the number of tilings in S that are invariant under some 
involution on S arising from a symmetry of the hexagon. See [161]. 

A different symmetry concerns the size parameter n, rather than the region be- 
ing tiled. The quantity sr which counts matchings of fortress graphs, is invariant 
under the substitution that replaces n by —n. The quantity 2n(n+1)/2, which counts 
matchings of Aztec diamond graphs, is invariant under the substitution that replaces 
n by —1—n. The quantity (@”+! — (—1/@)"*!)/./5, which counts matchings of the 
2-by-2n grid, is invariant under the substitution that replaces n by —2 —n, up toa 
predictable factor of +1. This phenomenon is called reciprocity, and it can be seen 
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in many exact enumerations. To cite a recent example, Ciucu and Lai [108, 37] have 
proved a conjecture of Matt Blum (problem 25 in [137]) and shown that, for positive 
integers a,b with b > 2a, the number of tilings of the hexagonal dungeon of type 
a,2a,b is 132° 14/2). This quantity is an even function of a; that is, it is invariant 
under the substitution that replaces a by —a. Likewise one notices that the polynomi- 
als that appears in formulas (7) through (10) in [150] (as well as many other formulas 
in that article, both proved and conjectural) are even functions of n. 

In some situations, reciprocity can be seen as a special case of a more general phe- 
nomenon linking two enumeration problems that yield respective integer sequences 
a\,a2,... and b} ,b2,...; the two problems are “mutually reciprocal” if (upon defin- 
ing dg,a_1,... and bo,b_1,... appropriately) one has a, = be_» and by = ac_y for 
some constant c. 

In the case of the 2-by-2n grid, and for other |1-dimensional matching problems, 
the beginnings of a satisfactory explanation for reciprocity can be found in [139] 
and [1]. However, for 2-dimensional tiling problems, nothing has been done. Since 
shuffling can be done in reverse, one might guess that the octahedron recurrence 
would provide the scaffolding for an explanation, via an extension of Kuo conden- 
sation to signed graphs like the ones considered by [1] (featuring various sorts of 
negative vertices and edges). Indeed, the rich octahedral symmetries of solutions to 
the octahedron recurrence call out for such a point of view, in which there would not 
be such a stark difference between “space” and “time,” let alone between the positive 
and negative time-directions. 

Although the introduction of signed graphs might seem like a conceptual extrav- 
agance, there is a very practical application they might have for the pattern-seeking 
combinatorialist. If one is looking for a recurrence relation that governs a one-sided 
sequence a1,d2,..., and if one suspects that the unknown recurrence relation would 
actually be satisfied by a two-sided sequence ...,d_2,a_1,d0,41,d2,... that extends 
the original one-sided sequence, and if moreover one has a principled guess for what 
the numbers ap,a_;,a_2,... are, then one could apply methods like the Somos ma- 
trix singularity test to a finite sequence of the form a_,,a_;+1,...,d0,---,4s—1,s- 
The success of the method hinges on r+ being large, so one wants to take r and 
s individually as large as possible; that is, the data will guide us toward the correct 
recurrence (assuming there is one!) if we give it as many terms as we can, in both the 
forward and reverse directions. 


9.8 Software 


The page http://jamespropp.org/software.html gives links to a number of 
resources that are likely to be useful to researchers conducting experimental work on 
matchings of bipartite planar graphs and associated tilings. 

The program vaxmaple (developed by Greg Kuperberg, David Wilson and my- 
self) allows one to count the matchings of a bipartite planar graph using the method of 
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Kasteleyn. One inputs a graph in VA X-format (described in [137]) and the program 
outputs a Maple program which, if fed into Maple, returns the number of matchings 
of the graph. 

David Wilson’s vaxmacs package gives the user all the power of vaxmaple and 
more in an easy-to-use interactive environment based on the popular editor emacs. 
The documentation for this package is contained in the program itself. You will need 
emacs-19.30 or later in order to get full use of vaxmacs. 

Harald Helfgott’s program ren, written in C, implements the generalized 
domino-shuffling algorithm of [140]. 

If you want to count matchings of some non-bipartite planar graph, Matt Blum’s 
TCL-program graph lets you define an arbitrary planar graph, and Ben Wieland’s 
perl-program planemaple (which graph “knows about’) lets you count the match- 
ings of this graph efficiently, exploiting Kasteleyn’s method. The program graph 
will also let you count matchings of non-planar graphs, but it does so by the brute- 
force method of finding them all, which will not be practicable when the graph is 
large. Both programs require the user to make some modifications to the program 
header. 

Several of the above programs come equipped with compatible side-programs 
for printing attractive pictures of tilings, with various configurable options. Some can 
even be used to compute conditional probabilities of sub-configurations in a random 
matching, or to generate a random matching. 

In addition to the above programs, which are intended as research tools, there are 
also programs of a more pedagogical nature. 

In the 1990s, Jason Woolever created a Java applet that is now available at 
http://jamespropp.org/cftp/, and that allows one to generate random tilings 
of a variety of regions by means of Coupling From The Past (or CFTP). CFTP is a 
variant of Monte Carlo methods, and gives an approach to bias-free sampling from 
various sets of combinatorial objects [142]. The speed of CFTP correlates directly 
with the speed at which a certain Markov chain mixes, namely, the chain in which 
one makes random local changes in the tiling by increasing or decreasing the height 
at a point. It may be noticed that CFTP is considerably slower for tilings of fortresses 
and dungeons than for the other cases that are presented. That is because random 
diabolo tilings of fortresses, unlike random domino tilings of Aztec diamonds or 
random lozenge tilings of hexagons, exhibit not only frozen domains and a liquid 
domain, but also a gaseous domain (in the terminology of [83]). In height-function 
terms, the height function is (up to small local fluctuations) flat throughout a large 
expanse of the tiling far away from the boundary, so that the graph of the height- 
function exhibits a broad plateau. This plateau can be at any of several heights, and it 
is very hard for the Markov chain to change the plateau-height from one of its likely 
possible values to another, since this requires changing the local height at O(n”) loca- 
tions, and any motion in this direction is likely to be reversed almost immediately by 
a kind of surface tension (entropic rather than energetic in nature). Thus there are bot- 
tlenecks in the Markov chain, and mixing is (or at least appears to be) exponentially 
slow. CFTP is therefore not an efficient way of generating random diabolo tilings of 
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fortresses; other methods (such as generalized domino-shuffling or the conditional 
probability method) must be used instead. The same is true for dungeons. 

In 2009 Ben Wieland modified the code to generate random alternating-sign ma- 
trices; see http: //nokedli.net/asm-frozen/. It should be mentioned that for 
the specific case of alternating-sign matrices of large order, CFTP is the only known 
efficient method of generating unbiased samples from the uniform distribution. 

For tilings of Aztec diamonds (or TOADs, as Hal Canary proposed calling them) 
there is an attractive Java applet jamespropp.org/toadshuffle/ created by Ca- 
nary that uses the original Kuperberg-Propp [53] version of domino-shuffling (with 
stages of annihilation, sliding, and creation) to iteratively generate uniformly random 
tilings of ever-larger Aztec diamonds. 


9.9 Frontiers 


Here are some directions in which further work is needed. 


1. It was mentioned above that ASMs and TSSCPPs are equinumerous; specifi- 
cally, both families of combinatorial objects are enumerated by the sequence 
1,1,2,7,42,429,.... This remains largely a mystery, as no bijective proof is 
known. 


In a similar but more tractable vein, it has been noticed that the sequence 
1,1,2,8,64,1024,... has arisen in several contexts [53] [46] [126] [114]. To 
the extent that all of these formulas have been given bijective proofs, one can 
provide bijections between the different models. However, some bijections are 
more natural than others, and one might hope to bring some precision to bear 
on the question of how similar, and how different, these various models are. 


2. As I mentioned in Section 9.7.1, as a step towards understanding the class of 
ultra-smooth enumerations of matchings, it would be good to classify the triply 
periodic solutions of the octahedron recurrence. A related project is to better 
understand all doubly periodic bipartite planar graphs. For any k, one can hope 
to classify all such graphs in which the fundamental domain contains k white 
vertices and k black vertices, up to deformations that preserve the combinato- 
rial structure. 


3. Section 9.7.5 mentions the role of symmetry. In that passage, we thought about 
starting with a finite graph and looking at matchings that possess invariance 
under some symmetry of the graph. One may take a different point of view 
and look at cofinite (or quasitransitive) group-actions on a finite graph, where 
we say an action on a graph is cofinite if there are only finitely many vertex- 
orbits and edge-orbits. If the infinite graph is for instance the square grid, then 
quotient graphs include tori, Klein bottles, and cross-caps, as well as more 
arcane orbifold-type quotients. The case of the torus (that is, enumeration of 
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matchings of a grid on a torus) is classical [78] and the case of the Klein bottle 
was settled by Lu and Wu [115] along with related problems in which the 
regions are given boundaries (ribbon graphs, Mobius strips, etc.). However, 
the case of the cross-cap remains unsettled. 


. Kenyon, inspired by earlier work on resistor networks and Ising models em- 


bedded in the complex plane, studied the bipartite planar dimer model on iso- 
radial graphs [81]. In that article Kenyon showed (partly answering the final 
question raised in [40]) that any periodic isoradial dimer model has entropy, 
which is the volume of a certain 3D ideal polytope. Li recently proved a confor- 
mal invariance theorem for this class of models [111]. The connection between 
dimer models and hyperbolic geometry is still a mystery. 


. We have seen a host of exact enumerative results coming from the dimer model 


on a variety of graphs, as well as a few exact enumerative results coming from 
the six-vertex model on a single graph, namely, the square grid. Might there be 
exact enumerative results for the six-vertex model on other 4-regular graphs, 
such as the Kagome (or star-of-David) lattice, or other statistical mechanics 
models of a similar kind, such as the twenty-vertex model on the 6-regular 
triangular lattice? 


We might try to move the theory of enumeration of tilings beyond the frame- 
work of dimer and ice-type models. An important step in this direction has 
been taken in the theory of square-triangle-rhombus tilings, mentioned at the 
end of Section 9.6. 


. It has been hard to get traction on the problem of counting monomer-dimer 


configurations in two (or more) dimensions, but other variants of dimers have 
been amenable to algebraic methods. Successful efforts in this direction in- 
clude the double dimer model [85] and the monopole-dimer model [3]. Like- 
wise, a variant of spanning trees that can be enumerated using a variant of the 
matrix tree theorem is cycle-rooted spanning forests [84]. 


If one is content with asymptotic results (as opposed to exact enumerations), 
and to work in the regime where the number of monomers is much smaller than 
the number of dimers, progress can be made; indeed, a great deal of interesting 
work has been done, much of it by Ciucu [28, 30, 32, 33, 24]. A guiding 
theme here is that correlational behavior of the model, in the bulk limit, mimic 
features of electrostatics. See also [57] and [52]. 


. Ribbon tilings (see Example 9.1.4 from Section 9.1) have not played much of a 


role in this chapter, but there has been some extremely interesting recent work 
on ribbon tilings of a special kind, namely “Dyck tilings.’ We will not de- 
fine the term or delve into the subject here, but we point the reader toward [88] 
and [89] as articles in this rapidly growing literature, as well as the article [153] 
showing the relevance of Dyck tilings to the study of Kazhdan-Lusztig poly- 
nomials. 
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8. 


10. 


As was mentioned in Section 9.1, Pak, Yang, and others have shown that tiling 
problems in which the allowed tiles and the region being tiled are all rectan- 
gles can be #P-complete. Even as we try to expand the domain of efficiently 
solvable tiling problems, we should try to find more and more constrained 
classes of tiling problems that, despite their strictures, still can be shown to be 
#P-complete. 


. By taking the g-analogue of the formula H (3n)H(n)* /H(2n)? for the number 


of rhombus tilings of a regular hexagon of side-length n and sending n to infin- 
ity, one recovers MacMahon’s formula for the number of unconstrained plane 


partitions: 
II 
n=l Sago 


One can think of this formula as g-enumerating a particular infinite set of 
rhombus-tilings of the plane. Let 7p be a tiling of the plane that divides the 
plane into 120-degree sectors, with no tiles crossing between one sector and 
another. Suppose T is a tiling of the plane that agrees with 7) outside some 
bounded region (call such a tiling cofinitely equivalent to 7). Then, interpret- 
ing the two tilings as surfaces, we can define the rank of T as the volume of 
the portion of space bounded between the two surfaces (normalized so that 
the volume of a cube is 1). MacMahon’s formula enumerates the tilings T by 
rank; that is, MacMahon’s product is equal to g'"*(), summed over all tilings 
T cofinitely equivalent to 7p. 


A beautiful “domino-analogue” of this result is the formula 


d+q@ Le 1 


Toa 


conjectured independently by Szendréi [163] and Kenyon [82] and proved by 
Young [174, 175]. One might ask whether other formulas of this kind await 
discovery. 


Recent work of Bouttier, Chapuy, and Corteel [15] creates a unifying context 
for results of this sort and more standard results in the theory of enumeration 
of tilings. Especially noteworthy is the development of the theory of vertex 
operators, introduced in [175]. 


It is natural to try to extend the existing theory of enumeration of tilings into 
higher dimensions. However, this may not be possible. For instance, consider- 
ing the fact that the double product 


counts lattice paths in a rectangle, and the fact that the triple product 


a Een ly oo Cem 


HII 
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counts plane partitions in a box (and their associated discrete spanning sur- 
faces), one might hope that the quadruple product 


a bed Lk+]—2 


Nn 


i=l j=lk=1l=1 TI 4 


might count some sort of solid partitions (associated with some sort of discrete 
3-surfaces in 4-space). However, the quadruple product is not even a whole 
number when a = b = c = d =2! (For recent work on enumerating solid par- 
titions, see [49].) 


Nonetheless, one can get some exact enumerative results relating to discrete 
2-surfaces in n-space. (This should not be too surprising, when one consid- 
ers that it is easy to count discrete 1-surfaces in n-space with fixed boundary 
using multinomial coefficients.) The first result along these lines was Elnit- 
sky’s enumeration of lozenge tilings of a semiregular octagon with side-lengths 
a,b,1,1,a,b,1,1 [54]: the number of tilings is 


2(a+b+1)!(a+b+2)!/alb!(a+2)\(b +2)! 


In the case of a semiregular octagon with side-lengths a,1,b,1,a,1,b,1, the 
enumerations have much larger prime factors, so no nice product formula is 
possible, but there is still a double sum formula found by Elnitsky; for more 
details, see [48]. It should be mentioned that for both of these two-parameter 
families of semiregular octagons, Stembridge’s g = —1 phenomenon has been 
proved to occur; in the a,b, 1,1,a,b,1,1 case the result is due to Elnitsky [54]: 
in the a,1,b,1,a,1,b,1 case it is due to Bailey [4] (Corollary 3.3 on page 25). 
See also [71]. 


One might also seek an exact formula for the number T7,, of lozenge tilings of 
a 2n-gon with all side-lengths equal to 1, but this appears to be a hard prob- 
lem. We know from [7] that lim(log T,)/n” < 7 but little is known about this 


sequence aside from asymptotics. 


Jumping off from this example into a general consideration of the growth rates 
of sequences that arise in enumerative combinatorics, let us say that a combinatorial 
sequence d1,da2,... (along with the combinatorial problem that gave rise to it) is of 
grade k if the logarithm of a, grows like n‘; that is, (logloga,)/logn converges to k. 
(For problems with more than one parameter, assume that all parameters are of the 
same order as n.) Combinatorics of grade 0 concerns sequences like the sequence of 
triangle numbers 1,3,6,10,.... Combinatorics of grade | concerns sequences like the 
Fibonacci and Catalan sequences; Examples 9.1.1 through 9.1.4 of Section 9.1 are 
all at grade 1. Combinatorics of grade 2 largely concerns genuinely two-dimensional 
tiling problems, many of which (like rhombus tilings of the a,b,c hexagon) can be 
recast as problems involving discrete 2-surfaces in three dimensions; more generally, 
rhombus tilings of a semiregular polygon with 27 pairs of parallel sides can be recast 
as discrete 2-surfaces in n dimensions. 
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In the past century, combinatorics moved from grade | to grade 2. However, 
we seem to be stuck there. Perhaps there is no such subject as grade 3 combinatorics 
(and perhaps, relatedly, there are no non-trivial exactly solvable 3-dimensional lattice 
models). Or perhaps there is such a subject, which we will begin to glimpse once we 
have gotten through more of the grade 2 curriculum and the teacher deems us ready 
for higher things. 
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10.1 Introduction 


A lattice path (path for short) is what the name says: a path (walk) in a lattice 
in some d-dimensional Euclidean space. Formally, a lattice path P is a sequence 
P=(Po,P1,...,P) of points P; in Z“. Figure 10.1 shows the lattice path ((0,0), (1,1), 
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Figure 10.1 
A lattice path. 


(2,1), (3,1), (3,2), (4,3)). The point Po is called the starting point and P, is called 
the end point of P. The vectors PoP, PiPs, oo PaiB are called the steps of P. 

Lattice paths have been studied for a very long time, explicitly at least since the 
second half of the 19th century. At the beginning stand the investigations concerning 
the two-candidate ballot problem [8, 123] (see the paragraph below Corollary 10.3.2 
in Section 10.3) and the gambler’s ruin problem [65] (see [38, Ch. XIV, Sec. 2] and 
Example 10.11.3 in Section 10.11). Since then, lattice paths have penetrated many 
fields of mathematics, computer science, and physics. The reason for their ubiquity 
is, on the one hand, that they are well-suited to encode various (combinatorial) ob- 
jects and their properties and, thus, problems in various fields can be solved by solv- 
ing lattice path problems. On the other hand, since lattice paths are, at the outset, 
reasonably simple combinatorial objects, the study of physical, probabilistic, or sta- 
tistical models is attractive in its own right. In particular, the importance of lattice 
path enumeration in non-parametric statistics seems to explain that the only books 
that are entirely devoted to lattice path combinatorics that I am aware of, namely [95] 
and [97], are written by statisticians. 

The aim of this chapter is to provide an overview of results and methods in lat- 
tice path enumeration. Since, in view of the vast literature on the subject, compre- 
hensiveness is hopeless, I have made a personal selection of topics that I consider of 
importance in the theory, the same applying to the methods I present here. 

Clearly, when one talks of “enumeration,” this comes in two different “flavors”: 
exact and asymptotic. In this chapter, I only rarely touch asymptotics, but rather 
concentrate on exact enumeration results. In most cases, corresponding asymptotic 
results are easily derivable from the exact formulas by using standard methods from 
asymptotic analysis. See [43] for “the” standard text on asymptotic methods in com- 
binatorial enumeration. 

In many cases, I omit proofs. The proofs that are given are either reasonably 
short, or they serve to illustrate a key method or idea in lattice path enumeration. If 
one attempts to make a list of the important methods in lattice path enumeration, then 
this will include the following. 


1. Generating functions (of course), in combination with the Lagrange in- 
version formula and/or residue calculus (see the second proof of Theo- 
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rem 10.4.5, the proof of Theorem 10.3.4, and the proof of Theorem 10.12.1 
for examples); 


2. bijections (they appear explicitly or implicitly at many places); 
3. reflection principle (see the proof of Theorem 10.3.1 and Section 10.18); 
. cycle lemma (see Section 10.4); 


. transfer matrix method (see the proof of Theorem 10.11.1); 


NH nm Ff 


. kernel method (see the proof of Theorem 10.12.2 and the paragraphs there- 
after); 


7. the path switching involution for non-intersecting lattice paths (see Sec- 
tion 10.13); 


8. manipulation of two-rowed arrays for turn enumeration (see Section 10.14); 


9. orthogonal polynomials, continued fractions (see Sections 10.9-10.11). 


We start with some simple results on the enumeration of paths in the d- 
dimensional integer lattice in Section 10.2. The sections that follow, Sections 10.3- 
10.7, discuss so-called simple lattice paths in the plane integer lattice Z*; these 
are paths in Z* consisting of horizontal and vertical unit steps in the positive direc- 
tion. While still staying in the plane integer lattice, beginning from Section 10.8, 
we allow three kinds of steps: Changing the geometry slightly by a rotation about 
45°, these are up-, down-, and level-steps. The case of Motzkin paths is intimately 
related to the theory of orthogonal polynomials and continued fractions. This link 
is explained in Sections 10.9-10.11. Section 10.12 provides a loose collection of 
further results for lattice paths in the plane integer lattice, with many pointers to 
the literature. The subsequent section, Section 10.13, is devoted to the theory of 
non-intersecting lattice paths, which is an extremely useful enumeration theory with 
many applications, particularly in the enumeration of tilings, plane partitions, and 
tableaux, but is also of great interest in its own right. Turn statistics are investigated 
in Section 10.14. Again, the original motivation comes from statistics, but more re- 
cent work, most importantly work on counting non-intersecting lattice paths by their 
number of turns, arose from problems in commutative algebra. Then we move into 
higher-dimensional space. Sections 10.15—10.17 present standard results for lattice 
paths in higher-dimensional lattices. How far one can go with the reflection princi- 
ple is explained in Section 10.18. The brief Section 10.19 gives some glimpses of 
q-analogues, including pointers to the connections of lattice path enumeration with 
the Rogers—Ramanujan identities. 

We conclude this introduction by fixing some notation that will be used consis- 
tently in this chapter. (It is in part inspired by standard probability notation.) Given 
lattice points A and E, a set S of steps (vectors), a set of restrictions R, and a non- 
negative integer m, we write 

Lm(A > E;S | R) (10.1) 
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for the set of all lattice paths from A to E with m steps, from S, which obey the 
restrictions in R. The lattice itself in which these paths are considered will be always 
clear from the context and is therefore not included in the notation. For example, the 
path in Figure 10.1 is in 


Ls((0,0) — (4,3); {(1,0), (0,1), (1,1) } |x 2y), 


where x > y indicates the restriction that the x-coordinate of any lattice point of the 
path is at least as large as its y-coordinate, or, equivalently, obeys the restriction to 
stay weakly below the diagonal x = y. 

Parts in (10.1) may be left out if we do not intend to require the corresponding 
restriction, or if that restriction is clear from the context. For example, the set of 
lattice paths from A to E with horizontal and vertical unit steps in the positive di- 
rection without further restriction will be denoted by L(A + E;{(1,0), (0,1)}), or 
sometimes even shorter, if the step set is clear from the context, L(A > E). 

When we consider weighted counting, then we shall also use a uniform nota- 
tion. Given a set .@ and a weight function w on .@, we denote by GF (.@;w) the 
generating function for .@ with respect to w, i.e., 


GF (M;w):= ¥ w(x). (10.2) 


xEM 


Finally, by convention, whenever we write a binomial coefficient tan it is as- 
sumed to be zero if k is not an integer satisfying 0<k <n. 


10.2 Lattice paths without restrictions 


In this short section, we briefly cover the simplest enumeration problems for lattice 
paths. If we are given a set of steps S, then the number of paths starting from the 
origin and using n steps from S is |S|”. If we are also fixing the end point, then we 
cannot expect a reasonable formula in this generality. 

However, in the case of (positive) unit steps such formulae are available. Namely, 
the number of paths in the plane integer lattice Z? from (a,b) to (c,d) consisting of 
horizontal and vertical unit steps in the positive direction is 
c+d-a ’). (10.3) 


c—a 


|L((a,b) > (c,d))| = ( 


since each path from (a,b) to (c,d) can be identified with a sequence of (c — a) 
horizontal steps and (d— b) vertical steps, the number of those sequences being given 
by the binomial coefficient in (10.3). 

More generally, for the same reason, the number of paths in the d-dimensional 
integer lattice Z¢ from a = (a1,a2,...,aq) to e = (€1,€2,...,€q) consisting of pos- 
itive unit steps in the direction of some coordinate axis is given by a multinomial 
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Figure 10.2 
Two paths illustrating formula (10.7). 


coefficient, namely 


IL( = ( Ye (ei—4ai) ) _ (x41 (e;—a;))! 
a—e)|= = ; 
Of Gj, e) Grieg — ag (e1 — a1)! (€2 —@2)!-+- (eg —ag)! 
(10.4) 
There is another special case, in which one can write down a closed form expres- 
sion for the number of paths between two given points with a fixed number of steps. 
Namely, the number of paths with n horizontal and vertical unit steps (in the positive 
or negative direction) from (a,b) to (c,d) is given by 


[Ln ((a,b) > (c,d); {(£1,0), (0,+1)})| = Cea) Caan) (10.5) 


See [60] and the references given there. 

If one considers other step sets then it may often be possible to obtain (non- 
closed) formulae by “mixing” steps. A typical example is the case where we con- 
sider lattice paths in the plane allowing three types of steps, namely horizon- 
tal unit steps (1,0), vertical unit steps (0,1), and diagonal steps (1,1). Let S = 
{(1,0),(0,1),(1,1)} be this step set. If we want to know how many lattice paths 
there exist from (a,b) to (c,d) consisting of steps from S, then we find 


UW ( ctd—-a-—b-k 
IL((a,b) + (c4);5)| = 0) ane a (10.6) 


since, if we fix the number of diagonal steps to k, then the number of ways to mix k 
diagonal steps, c—a—k horizontal steps, and d — b —k vertical steps, is given by the 
multinomial coefficient, which represents the summand in (10.6). In the special case 
where (a,b) = (0,0), the corresponding numbers are called Delannoy numbers, and, 
if (c,d) = (n,n), central Delannoy numbers. 

As a first excursion to weighted counting, we consider the generating function for 
lattice paths in Z? from A = (a,b) to E = (c,d) consisting of horizontal and vertical 
unit steps in the positive direction, in which each path is weighted by gi? ), where 


594 Handbook of Enumerative Combinatorics 


a(P) denotes the area between the path and the x-axis (with portions of the path that 
lie below the x-axis contributing a negative area). More precisely, the area a(P) is the 
sum of the heights (abscissa) of the horizontal steps of P. For example, for the left- 
hand path in Figure 10.2 we have a(.) = 1+3+3+4 = 11, while for the right-hand 
path we have a(.) = (—1)+(—1) +142 = 1. It is then straightforward to check (by 
induction on the length of paths) that 


c-a) |C+d—a—b 
GF (L((a,b) > (c,d));q")) =?" a ae | (10.7) 
q 
where [“'¢=7? | | denotes the g-binomial coefficient defined by 


k egy gt) hag Bh (Tag) 


and [;|, = 0 if k < 0. This result connects lattice path enumeration with the theory 
of integer partitions. What we have computed in (10.7) is equivalent to the classical 
result that the generating function for integer partitions with at most k parts, each of 
which is bounded above by n is given by [eee | ¢ We shall say a little bit more about 


A _ (1—q")(—q""')---—4@) 
ge. eh 


q-counting in Section 10.19. The reader is referred to [2] for an excellent survey of 
the theory of partitions. 


10.3. Linear boundaries of slope 1 


Next we want to count paths from (a,b) to (c,d), where a > b and c > d, which 
stay weakly below the main diagonal y = x. So, what we want to know is the num- 
ber |L((a,b) — (c,d) |x>y) |. This problem is most conveniently solved by the so- 
called reflection principle most often attributed to André [1]. However, while André 
did solve the ballot problem, he did not use the reflection principle. Its origin lies 
most likely in the method of images of electrostatics, see Sections 2.3—2.6 in [64]. 


Theorem 10.3.1 Let a > b and c > d. The number of all paths from (a,b) to (c,d) 
staying weakly below the line y = x is given by 


|L((a,b) + (c,d) |x> i aera = a een | (10.8) 


Proof. First we observe that the number in question is the number of all paths from 
(a,b) to (c,d) minus the number of those paths that cross the line y = x, 


|L((a,b) > (c, d)|x>y)|=|L((a,b) > (c,d))| 
_ IL(( a,b) — (c,d) | x Z y at least once) |. (10.9) 
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Figure 10.3 
A path P crossing the line x = y. 


By (10.3) we already know |L((a,b) — (c,d)) |. The reflection principle shows that 
paths from (a,b) to (c,d) that cross y = x are in bijection with paths from (b— 1,a+ 
1) to (c,d). This implies 


|L((a,b) — (c,d) |x Zy at least once) | = IL((b- l,a+1)—> (c,d))| ; 


Hence, using (10.3) again, we establish (10.8). 

The claimed bijection is obtained as follows. Consider a path P from (a,b) to 
(c,d) crossing the line y = x. See Figure 10.3 for an example. Then P must meet 
the line y=x-+1. Among all the meeting points of P and y= x-+1, choose the 
right-most. Denote this point by S. Now reflect the portion of P from (a,b) to S in 
the line y = x+ 1, leaving the portion from S to (c,d) invariant. Thus we obtain a 
new path P’ from (b—1,a+1) to (c,d). To construct the reverse mapping we only 
have to observe that any path from (b— 1,a+ 1) to (c,d) must meet y = x+ 1 since 
(b—1,a+1) and (c,d) lie on different sides of y =x + 1. Again we choose the right- 
most meeting point, denote it by S, and reflect the portion from (b— 1,a+1) to S 
in the line y =x + 1, thus obtaining a path from (a,b) to (c,d) that meets the line 
y =x-+1 or, equivalently, crosses the line y = x. | 


In particular, for a = b = 0 we obtain the following compact formula. 


Corollary 10.3.2 [fc > d we have 


ame) (10.10) 


IL((0,0) > (ed) |x2y)|=— alg 
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and 


it 49 
|Z((0,0) + (@,n) [x2 y)| = @ (10.11) 


The numbers ci4 (ee) are called ballot numbers since they give the answer 
to the classical ballot problem, which is usually attributed to Bertrand [8], but was 
actually first stated and solved by Whitworth [123]. The problem is stated as follows: 
In an election candidate A received c votes and candidate B received d votes; how 
many ways of counting the votes are there such that at each stage during the counting 
candidate A has at least as many votes as candidate B? By representing a vote for A 
by a horizontal step and a vote for B by a vertical step, it is seen that the number 
in question is the same as the number of lattice paths from (0,0) to (c,d) staying 
weakly below y = x. This number is given in (10.10). More about ballot problems 
appears in Sections 10.12 and 10.18. 

The numbers ran (@) are called Catalan numbers [27, 28]. However, they have 
been considered earlier by Segner [106] and Euler [37], and independently even ear- 
lier in China; see the historical remarks in [101] and [112, p. 212]. They appear in 
numerous places; see [112, Ex. 6.19], with many more occurrences in the addendum 
[113]. 

An iterated reflection argument will give the number of paths between two diag- 
onal lines. 


Theorem 10.3.3 Leta+t>b>a+s andc+t>d>c+s. The number of all 
paths from (a,b) to (c,d) staying weakly below the line y = x +t and above the line 
y=x+s is given by 


|L((a,b) + (c,d) |x+t>y>x+4+5)| 
hal ct+d-—a-—b )-( c+d—a-—b )). (10.12) 
fa \ Noe hrs 2) c—b—k(t—s+2)4+t+1 

Since this is (as well as Theorem 10.3.1) an instance of the general formula 
(10.145) for the number of paths staying in regions defined by hyperplanes, we omit 
the proof. 

The formula in Theorem 10.3.3 is very convenient for computing the number 
of paths as long as the parameters are not too large. On the other hand, it is of no 
use if one is interested in asymptotic information, because the summands on the 
right-hand side of (10.12) alternate in sign so that there is considerable cancellation. 
However, with the help of little residue calculus, the formula can be transformed into 


a surprising formula featuring cosines and sines, from which asymptotic information 
can be easily read off. 


Theorem 10.3.4 Leta+t>b>a+s andc+t>d>c+s. The number of all 
paths from (a,b) to (c,d) staying weakly below the line y = x +t and above the line 
y=x+s is given by 


Lattice Path Enumeration 597 


|L((a,b) - (c,d) |x+t>y>x+5)| 


[(t-s+1)/2] 4 ak c+d—a—b 
= y (200s ) 
eye (e-St2 t—s+2 


t—s+2 t—s+2 


) (10.13) 


Proof. — Trivially, the binomial coefficient (7) is the coefficient of z~! in the Laurent 


series 
(1+z)" 
Zeer 


Thus, the sum (10.12) equals the coefficient of 2 an 


es a 4 z)etd-anb k(t—s+2) (1 4 z)eta-a-b zk(t—s+2) 


dX ze—a+(c+d—a—b)(t—st2)+1 — ze—b+t+(c+d—a—b) (t—s+2)+2 


Gz) (1z)te-2-6 
ze a+(c+d—a—b)(t s+2)+1(] — gf Ss 2) ze b+t+(c+d—a—b)(t s+2)+2(] — gf s+2\ 


(14z)¢td-a aG c+d+a ir anaes cas 


- z(ctd—a—b)/2+(ctd—a—b)(t s+2)+1(] — zt-s+2) : (10.14) 


(In the second line we used the formula for the geometric series. It can be either 
regarded as a summation in the formal sense, or else one must assume that |z| < 1.) 
Equivalently, the sum (10.12) equals the residuum of the Laurent series (10.14) at 
z = 0. Now consider the contour integral of (10.14) (with respect to z, of course) 
along a circle of radius r around the origin. It is a standard fact that in the limit 
r —> co this integral vanishes, because the integrand (10.14) is of the order O(1/27). 
Therefore, by the theorem of residues, the sum of the residues of (10.14) must be 0, 
or, equivalently, the residuum at z = 0, which we are interested in, equals the negative 
of the sum of the other residues. As the other poles of (10.14) are the (t —s+2)th 
roots of unity different from 1, we obtain 


Inik \ ct+d—a—b ik R ik . 
ae (1+er) (er2( c+d+a—b) _ 4S (—ct+d—a+b—21 ”)| 


Qnik_/ c+d—a—b 2nik ; 
k=1 Pec ae z— +1) (-(¢-5+2)enFa- 0) 


t—s+1 Peay: Say eae 3 
= y = reg orth (-e+d-t-1) 
t—s+2 t—s+2 


k=1 
(erate b+t+1) _ 4 IG (a ca) 


for the sum (10.12). Now, in the last line, we pair the kth and the (t —s+2—k)th 
summand. Thus, upon little manipulation, the above sum turns into 
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[(t-s+1)/2] 1 ak c+d—a—b 
cos 
aa 3) 


k=l 
Rik (c—d+t+1) Rik (c—d+t+1) Zik (a—b+t+1) BK, (a—b+t+1 
-( ee Hsp" TET?) _ @ts42 © : era (a Th) @7 TH s¢2 4-99 ) > 


Clearly, this formula is equivalent to (10.13). a 


From the generating function formula given in Section 10.11 (see Exam- 
ple 10.11.2), one can see that this asymptotic formula comes from Chebyshev poly- 
nomials. 


10.4 Simple paths with linear boundaries of rational 
slope, I 


When we want to count simple lattice paths (recall the meaning of “simple” from 
the introduction) in the plane bounded by an arbitrary line y = kx +d, k,d € R, the 
reflection principle obviously does not help, since the reflection of a lattice path in 
a generic line does not necessarily give a lattice path. In fact, a solution in form of 
a determinant can be given when the boundary is viewed as a special case of a set 
of general boundaries (see Section 10.7, Theorem 10.7.1; another solution was pro- 
posed by Takacs [118], which is of similar complexity as it involves the solution of 
a large system of linear equations). However, there are cases where simpler expres- 
sions can be obtained, and these are discussed in this section. All of them can be 
derived from a very basic combinatorial lemma, the so-called cycle lemma, which 
exists in several variations. 

The first case we discuss is the enumeration of simple lattice paths from the origin 
to a lattice point (r,s), with r and s relatively prime, which stay weakly below the line 
connecting the origin and (7,5). 


Theorem 10.4.1 Let r and s be relatively prime positive integers. The number of all 
paths from (0,0) to (r,s) staying weakly below the line ry = sx is given by 


1 ‘ 
2((0.0) > (rs) | «> 9) = —("*"). (10.15) 
r+s r 
Remark 10.4.2 The numbers in (10.15) are nowadays called rational Catalan 
numbers (cf. [4]), the Catalan numbers being the special case where r = n and 
s=n+l. 


The above result follows easily from a form of the cycle lemma, which is known 
in the statistics literature as Spitzer’s lemma [110]. 


Lemma 10.4.3 (Spitzer’s Lemma) Leta, ,a2,...,ay be real numbers with the prop- 
erty that a, +a2+-+:-+ay =0 and no other partial sum of consecutive a;’s, read 
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Figure 10.4 
An instance of Spitzer’s lemma. 


cyclically (by which we mean sums of the form aj + aj.) +++++ a, with j <k and 
k—j <N, where indices are interpreted modulo N), vanishes. Then there exists a 
unique cyclic permutation a;,dj+1,...,4N,@1,...,@i—1 with the property that for all 
j=1,2,...,N the sum of the first j letters of this permuted array is non-negative. 


Remark 10.4.4 This lemma could be further generalized by weakening the above 
assumption to demand that K partial sums of consecutive a;’s, read cyclically, of 
minimal length vanish, with the conclusion that there be K cyclic permutations with 
the above non-negativity property. 


Proof. We interpret the real numbers a; as steps of a path (although not necessarily 
of a lattice path), by concatenating the steps (1,a1), (1,a2), ..., (1.ay) to a path 
starting at the origin. See the left half of Figure 10.4 for a typical example. 

Since the sum of all a;’s vanishes, the end point of the path lies on the x-axis. 
We identify this end point with the starting point (located at the origin), so that we 
consider this path as a cyclic object. 

By the non-vanishing of cyclic subsums, there is a unique point of minimal 
height, A say. (This may also be the starting/end point, which we identified.) In the 
figure this point is marked by a thick dot. Now “permute” the path cyclically, that 
is, take the portion of the path from A to the end, and concatenate it with the initial 
portion of the path until A. See the right half of Figure 10.4 for the result in our ex- 
ample. Obviously, the new path always lies strictly above the x-axis, except at the 
beginning and at the end. This identifies the cyclic permutation of the a;’s with the 
required property. | 


Proof of Theorem 10.4.1. We consider all paths from (0,0) to (r,s). There are 
("**) such paths. Given a path P from (0,0) to (r,s), we consider the sequence 
a1,42,..-,a;++5, where a; = s if the ith step of the path is a horizontal step, and a; = —r 
if the ith step of the path is a vertical step. Since r and s are relatively prime, no 
cyclic subsum of the a;’s, except the complete sum, can vanish. The cycle lemma in 
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Lemma 10.4.3 then implies that, out of the r+ s cyclic “permutations” of the path P, 
there is exactly one that stays (weakly) below the line sx = ry. Thus, there are in total 
a ("**) paths with that property. | 

The next case where a closed form formula can be obtained (partially overlapping 
with the result in Theorem 10.4.1) is when counting lattice paths from (0,0) to (c,d), 
which stay weakly below the line x = Uy, where [ul is a positive integer. Of course we 
have to assume c > Ud. There are two conceptually different standard approaches to 
obtain the corresponding result: application of another version of the cycle lemma 
(see Lemma 10.4.6), respectively generating functions combined with the use of the 
Lagrange inversion formula. 


Theorem 10.4.5 Let t be a non-negative integer and c > tld. The number of all 
lattice paths from the origin to (c,d) that lie weakly below x = Ly is given by 


_crHd+1 (et+d+1 
|L((0,0) — (c,d) |x > py)| = c+d+l ( d ), 


(10.16) 

This result is essentially equivalent to the cycle lemma due to Dvoretzky and 
Motzkin [36]. It has been rediscovered many times; see [35] for a partial survey and 
many related references, as well as [112, Lemma 5.3.6 and Example 5,3,7]. 


Lemma 10.4.6 (Cycle Lemma) Let ul be a non-negative integer. For any sequence 
P1p2---Pmtn of m 1’s and n 2’s, with m > wn, there exist exactly m— pn cyclic 
permutations pipi+1---PmtnP1---Pi-1, | <i <m-+n, that have the property that 
for all j =1,2,...,m-+n the first j letters of this permutation contain more 1’s than 
Lt times the number of 2’s. 


Proof. A sequence pi p2...Pm+n Of m 1’s and n 2’s can be seen as a lattice path 
from (0,0) to (m,n) by interpreting the 1’s as horizontal steps and the 2’s as vertical 
steps. Cyclically permuting p12... Pm+n Means to cut the corresponding lattice path 
into two pieces and put them together in exchanged order, thus obtaining a new lattice 
path from (0,0) to (m,n). Finally, the property that in each initial string of a sequence 
the number of 1’s dominates (i.e., is larger than) fu times the number of 2’s means that 
the corresponding lattice path stays strictly below the line x = ty, with the exception 
of the starting point (0,0). 

For the proof of the lemma interpret p1 p2...Pm+n as a path, as described before, 
and join a shifted copy of this path at the end point (m,n), another shifted copy at 
(2m,2n), etc. Figure 10.5 shows an example with up = 2, m = 9, n = 3. The path 
P corresponds to the sequence 121111122111. Cyclic permutations of pip2...Pm+n 
correspond to cutting a piece of m+n successive steps out of this lattice path struc- 
ture. Then imagine a sun to be located in direction (u, 1) illuminating the lattice path 
structure. A cyclic permutation will satisfy the dominance property for each initial 
string if and only if the first step of the corresponding lattice path is illuminated. In 
Figure 10.5 the illuminated steps are indicated by thick lines. It is an easy matter of 
fact that of any m+n successive steps there are exactly m— un illuminated steps. 
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Figure 10.5 
An instance of the cycle lemma with u = 2, m= 9, andn = 3. 


Therefore out of the m+n cyclic permutations of p1p2...Pm+n there are exactly 
m— [ln cyclic permutations having the dominance property for each initial string. I 


First proof of Theorem 10.4.5. | We want to count paths from (0,0) to (c,d) staying 
weakly below x = Ly. To fit with the cycle lemma we adjoin a horizontal step at the 
beginning and shift everything by one unit to the right. Thus we are now asking for 
the number of paths from (0,0) to (c+ 1,d) staying strictly below x = py, except for 
the starting point (0,0). Now one applies Lemma 10.4.6 with m= c+ 1 andn=d: 
given a path P from (0,0) to (c+1,d), exactly m— un =c+1-—vwd of its cyclic 
“permutations” satisfy the property of staying strictly below x = jy, except for the 
starting point (0,0). Thus, the total number of paths from (0,0) to (c,d) with that 
property is given by (10.16). a 


For instructional purposes, we also present the generating function proof. 


Second proof of Theorem 10.4.5. The generating function proof works in two steps. 
First, an equation is found for the generating function of those paths that return in the 
end to the boundary x = Uy. Then, in a second step, paths ending arbitrarily are de- 
composed into paths of the former type, leading to a generating function expression 
in terms of the earlier generating function to which the Lagrange inversion formula 
is applicable. 

Let P be a path in L((0,0) + (ud,d) | x > py) (see Figure 10.6). For | = 
0,1,...,u4—1, the path P will meet the line x = y+/ (which is parallel to our 
boundary x = ty) somewhere for the last time. Denote this point by S;. Clearly, the 
path P must leave S; by a horizontal step, which we denote by sy», for short. This gives 
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Figure 10.6 
The path P and the line x = Ly. 


us a unique decomposition of P of the form 
P= PospPisy of -SpPusy, 


where Po is P’s portion from the origin up to So, P; is P’s portion from the point 
immediately following So up to S}, etc. By s, we denote the final vertical step. All 
the portions P; (when shifted appropriately) belong to L((0,0) + (un,n) | x > by) 
for some n. Let 

& = (JL((0,0) — (un,n) | «> py). (10.17) 


n>0 


Then we have the following decomposition: 
L= {e}U ((Losn)* Dosy)- 


Here, as always in the sequel, € denotes the empty path. 
By elementary combinatorial principles, this immediately translates into a func- 
tional equation for the generating function 


Fo(z) = ¥ |L((0,0) > (un,n) | x > wy)|z" 


n>0 


for Zp (note that the summation index n records the vertical height of the end point 
of paths), namely 


Fo(z) =1+2Fo(z)4t!. (10.18) 

If we write Fo(z) = 1 + Go(z), then Equation (10.18) in terms of the series Go(z) 
reads Go(2) 
o(z 

me 10.19 

(+Go(@yet * nee 


which simply says that Go(z) is the compositional inverse of z/(1 +z)#+1. 
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Figure 10.7 
The more general problem. 


Turning to the more general problem, consider a lattice path P in |L((0,0) —> 
(ud +k,d) |x > py)| (see Figure 10.7). For / =0,1,...,k—1 the path will meet the 
line x = uy +/ somewhere for the last time. Denote this point by S;. Clearly, the path 
P must leave S; by a horizontal step, for which we again write s;. This gives us a 
decomposition of P of the form 


P= PospPiSp..-SnPe, 


where Py is the portion of P from the origin up to So, P; is the portion of P from the 
point immediately following So up to S;, and so on. Observe that again the portions 
P; belong to Y (being defined in (10.17)). Let 


L_= (J L((0,0) + (unten) |x> my). 


n>0 
Then we have the following decomposition: 
Be=(Asi) 


This translates again into an equation for the corresponding generating function 


F(z) == ¥. |L((0,0) > (un+k,n) |x > py) |z" 


n>0 
for Y,, namely into 
F(z) = Fo(z)**1 = (1+ Go(z))*1. 


We noted above that Go(z) is the compositional inverse of z/(1+z)*!. Therefore, 
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we may apply the Lagrange formula (see [112, Corollary 5.4.3]; for the current pur- 
pose, we have to choose H(z) = (1+z)**!, f(z) =z/(1+z)#*! there). This yields 


ive (1+z)nH+) 
L((0,0) > (un +k,n) |x > my) = (R41) (12h 
_k+1 (pntk+n 
on n—-1 
_ k+1 un+k+n+1 
~ pn+k+n+1 n ; 
which turns into (10.16) once we replace un +k by c and n by d. a 


In particular, for u = 1 the generating function Fo(z) can be explicitly evaluated 
from solving the quadratic equation (10.18). In the case where the paths return to the 
boundary x = y, ie., where (c,d) = (n,n), this gives the familiar generating function 
for the Catalan numbers (compare with the second paragraph after Corollary 10.3.2) 
era pie cea (10.20) 

2z 


n>0 


More generally, if 4 is kept generic and (c,d) = (n,n) (that is, we consider 
again paths that return to the boundary), then the formula on the right-hand side 
of (10.16) becomes mel (oe. These numbers are now commonly called FuB- 
Catalan numbers, cf. [3, pp. 59-60] for more information on their significance and 
historical remarks. 

So far we only counted paths bounded by x = tty where the starting point lies on 
the boundary. If we drop this latter assumption and now want to enumerate all paths 
from (a,b) to (c,d) staying weakly below x = Ly, there is still an answer, although 
only in terms of a sum. In fact, we can offer two different expressions. Which of 
these two is preferable depends on the particular situation, to be more precise, on 
which of the numbers (a/ — b) or (d —a/) being larger (see Figure 10.8 for the 
pictorial significance of these numbers). Though the proof for the first expression 
is rather straightforward, the proof for the second expression is more difficult. The 
result below was first found by Korolyuk [73]. It is a special case of an even more 
general result of Niederhausen [98, Sec. 2.2] on the enumeration of simple paths with 
piecewise linear boundaries, which we will discuss in Section 10.6. 


Theorem 10.4.7 Let be a non-negative integer, a > Ub and c > Ud. The number 
of all lattice paths from (a,b) to (c,d) staying weakly below x = [Ly is given by 


cC—a 


(a,b) > (esd) |x > 1) =( 


d i(u+1)—a—b-1 c—pd+1 ct+d—i(u+1)+1 
i-b c+d—i(u+l1)+1 d—i 


eae 


i=|a/p|+1 
(10.21) 
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x= py 


d—a/u 


a/u—b 


Figure 10.8 
The significance of the numbers (a/ — b) and (d—a/). 


and also 


|L((a,b) — (c,d) |x > by) | = 


l 


|a/u|—b 

_ 1) a—u(b+i) 
i ae ae 
c—uUd+1 eae ee 
‘c+d—(u+lb+i+1 d—b-i 


): (10.22) 


Proof of (10.21). | The number of paths in question equals the number of all paths 
from (a,b) to (c,d) minus those paths that cross x = Uy. To count the latter observe 
that any path crossing x = [Ly must meet the line x = pry — 1, and for the last time in 
some point (f4i— 1, i) where |a/p| +1 <i<c. Fix such an i, then the number of all 
these paths is 


|L((a,b) + (ui-1,i))|-|L((ui,’) > (c,d) |x > wy). 


We already know the first number due to (10.3), and we also know the second number 
due to (10.16), since a shift in direction (—pi,—i) shows that the second number 
equals |L((0,0) + (c— Hi,d —i) |x > py). | 


Proof of (10.22). This is the special case of Theorem 10.6.1 where m = 2, by = 
v, =0, 91 = |a/M] —), bo =, V2 = Wb—a. 

For a different, direct proof, in the sum in (10.21) replace the index i by i+ b; the 
new index then ranges from |a/u| —b+1 tod —b; extend the sum to all i between 
0 and d — b, thereby adding a partial sum where i ranges from 0 to |a/u| — b; the 
former sum can be evaluated by means of a convolution formula of the Hagen—Rothe 
type (cf. [54, Eq. (11)]), and the result is the binomial coefficient (eee y | | 


c— 
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Enumeration of lattice paths in the presence of several linear boundaries can in 
the best cases be solved by an iterated application of the reflection principle; see 
Section 10.18 for the most general situation where the reflection principle applies. 
But, if it does not apply (which, in a random case, will certainly be so), then the 
enumeration problem will be very challenging. Usually, one cannot expect to find a 
useful exact formula (but see Section 10.7), and will instead investigate asymptotic 
behaviors. This is still quite challenging. The reader is referred to [22, 68, 69] for 
work in this direction. 


10.5 Simple paths with linear boundaries with rational 
slope, II 


In Section 10.4 we considered lattice paths bounded by a line x = yy, with pf a non- 
negative integer. Now we want to consider a more general linear boundary of the 
form vx = Uy, where v, are non-negative integers. We describe a generating func- 
tion approach, due to Sato [104], which works for a large class of cases. Alternative 
solutions, which work in all cases, in the form of a determinant, can be given as a 
special case of a set of general boundaries. These are discussed in Section 10.7, see 
in particular Theorem 10.7.1. 

The problem that we want to attack here is to enumerate all lattice paths from 
an arbitrary starting point to an arbitrary end point staying weakly below the line 
vx = Uy, where v and w are positive integers. A simple shift of the plane shows 
that this is equivalent to enumerating paths from the origin to an arbitrary end point 
staying weakly below vx = Ly — p, for an appropriate p. Without loss of generality 
we may assume in the sequel that v < yl. For the approach of Sato, this is the more 
convenient formulation of the problem. The idea is to introduce v x v matrices that 
contain the path numbers we are looking for. More precisely, define the v x v matrix 
W(z;c,p) by 


W(z;c,p) := (w(ac+g,p 4h) peng is (10.23) 
where 
w(z3c,p) = y w(n;c,p)z”, (10.24) 
Lun+c=p (mod v) 
with 
|L((0,0) + (=? -n) | vx >uy—p)|, un+c=p (mod v) 
; and un+c>p, 
rs ae (aamraao F Lin+c = p (mod v) 
and un+c<p. 
(10.25) 


So, what the matrix W(z;c,p) contains is generating functions of the path numbers 
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w(n;c, p) that we want to know. The definition of w(n;c,p) for un+c < p (in which 
case there cannot be any paths from (0,0) to (# mieoP ,n)) is just for technical conve- 
nience. Basically, the matrix W(z;c,p) is 


—p-h 
L((0,0) + (***48 E an) 


z') . (10.26) 
0<g,h<v—1 


The following theorem of Sato [104, Theorem 1] tells how to compute W (z;c,p). 


ricoo)=( 


LUn+c+g—h=p (mod v) 


vx > Ly — p —h) 


Theorem 10.5.1 Let 


va ((-1) sega tri) (H0(2)s---4v-1(2))) (10.27) 


0<g,h<v—1 


where 


S(@,18) (UO, -- .,Uy—1) 


= y Wig, (Z) Mig (Z) +++ iy (Z)Ujy (Z) +++ jg (Zz), (10.28) 


V—-1Big 2ig-1 2-4 <ji<<jpsv—l 
uj(z) being defined by 


: 1 ty (1+ ) : n 
a (2zil/v) V Vv 2nilu/V 
wile) e ees ceenclt (z¢ ) 


1=0,1,...,v—1. (10.29) 


Furthermore, let 


vce) ( : gs) (10.30) 
pl=p—g+h (mod v) 0<g,h<v—1 


LIi<p—gth 
Then, for any non-negative integers c, P, V, L, with V < LU, we have 
W(z:c,p) =M(z;c,p) P(z;p). (10.31) 


Note 10.5.2 Note that s(q ))(uo,--.,4v—1) is a Schur function of hook shape (cf: 
[89, Ch. I, Sec. 3, Ex. 9]). 


It might be useful to discuss an example, in order to illustrate what this is all about. 


Example 10.5.3 We take v = 2, pp = 3. So, by (10.25), the quantity w(n;c,p) rep- 
resents the number of all lattice paths from (0,0) to ((3n+¢—p)/2,n) which stay 
weakly below the line 2x = 3y — p, where c= 3n—p (mod 2), i.e., c=n+p (mod 2). 
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By definition (10.23), we have 


Wizc,p)= (w(zc+g,p aa) herrer 


_( we,p) w(zsc,p +1) 
~ Aw(e+lp) waetlpt+i))- 


Using (10.31), this can be written as 
W(z;c,p) = M(z;c,2) P(z;p), 


where 


y _ ~HI y l-yl 
H/=p (mod vy( i? i uN pl=p+1 (mod y(-bi(Pr i vy! 


Li<p bl<p+1 
1)F(@- ben) 2 1) (@-HDIY) 2 


ul=p-1 (mod we Lni=p (mod Ge 
HI<p—1 HI<p 
by (10.30), and 
MGca= ue eet ee) 
5(c42,1)(uo(Z),41(Z)) — 8(c42) (wo(z), 1 (2) 
by (10.27), with 5p) (uo(Z),ui(z)) being defined in (10.28), and 
jr 714s 
uj(z) =(—1)') Gage 


50 1+5n n 


as given in (10.29). 
So, in particular, in case that c = p = 0 the matrix ®(z;0) is the 2 x 2 identity 
matrix, and so we have 


: _ (w(z;0,0)  w(z;0, 1) 
Le Ga Aah 


eee —up(z)ur (2) ug(2) +1 (2) 
a (ae eo ji ae) | 


Whence, for even n the number of all lattice paths from (0,0) to (3n/2,n) which stay 
weakly below the line 2x = 3y, equals 


|E((0,0) — (3.x) | 2x > 3y)| = (2")w(z0,0) 
Sgt 1 (5431 I 5 +3(n-1) 
=v (? een ne i; 


and for odd n the number of all lattice paths from (0,0) to ((3n—1)/2,n) which stay 
weakly below the line 2x = 3y — 1, equals 


3n—1 ub : 2 zee 
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Sato [104] also derived a result of similar type for two parallel linear boundaries 
with rational slope. To be precise, we want to enumerate all lattice paths from an 
arbitrary starting point to an arbitrary end point staying weakly below a given line 
vx = Hy — p and above another given line vx = Uy+o, where L,V,p,0 are non- 
negative integers. Again, without loss of generality we may assume that v < pw and 
that the starting point is the origin. 

Following the approach we have taken earlier, we define the v x v matrix 
T (z3c,p,0) by 


T(z:¢,p,0):= (t(et+g,P+h,O—f)) gc ancy) (10.32) 
where 
t(z3c,p,0) = y t(nsc,p,o)z", (10.33) 
Lun+c=p (mod v) 
with 


|L((0,0) > (*4-*,n) | wy +o > vx> y—p)|, 
Ln+c =p (mod v) 
t(n3c,p,0) = and un+c>p, 


Ln+c =p (mod v) 


and un+c <p. 
(10.34) 


Similar to the one boundary case, what the matrix T(z;c,p,0) contains is generating 
functions of the path numbers t(n;c,p,0) that we want to compute. The definition 
of t(n;c,p,0) for un+c < p (in which case there cannot be any paths from (0,0) to 
(Peer n)) is just for technical convenience. Basically, the matrix T(z;c,pP, ©) is 


—p—h 
z((0,0) + (Mee P= 9) 


T(ae,p.0) =( y 


pun+c+g—h=p (mod v) 


py+o—h>vs> wy-p-8)|:) . (10.35) 


0<g,h<v— 


The following theorem of Sato [104, Theorem 4] tells us how to compute 
T (z3c,p,0). 


Theorem 10.5.4 For any non-negative integers c, P, 0, V, LL, with V < LL, we have 
T (z:c,p,6) = ®(z.p +o+1—c—p)O !(z;3p +041) P(z;p), (10.36) 
where ®(z; 0) is given by (10.30). 


Example 10.5.5 As an illustration, let us again consider the case V = 2, U = 3. By 
(10.34), the quantity t(n;c,p,0) represents the number of all lattice paths from (0,0) 


610 Handbook of Enumerative Combinatorics 


o ((3n+c—p)/2,n) which stay weakly below the line 2x = 3y — p and above the 
line 2x = 3y +0, where c = 3n—p (mod 2), i.e, c=n+p (mod 2). 
By definition (10.32), we have 
T(z5c,p,0) = (t(ae+g,p +h,O=h)) yconci 
a t(z3c,P,0) iHecp+ loa) 
ASL pal cecil pak los bye 


By Theorem 10.5.4, this can be written as 


= (opto scl). ogipto—e) 
r(aep.0)=(6 (z3p +o—c—2) Rear) 
. rte (sp +o+2)\" 
oe o(z3p +o+1) 
(zp +1) 
: Gan : nan ras 7 (10.37) 
where 
$(z:a) = y caylee 
Ll=a (mod v) : 


LUl<a 


In particular, if c= 6 and p = 0, so that we are, for example, interested in the 
number of all lattice paths from (0,0) to ((3n+c)/2,n) that stay weakly below the 
line 2x = 3y and above the line 2x = 3y +c (the reader should observe that this means 
that the starting point is on the first line whereas the end point is on the second line), 
then our formula (10.37) reduces to 


: if te,0,¢) t(z;c,1,c—1) 
ey as ee t(ge+1,1,¢—1) 


= 1 eg a) 
~ §2(ze+1)— o(zc)6(z;e +2) 0 0 , 
Thus we obtain 
t(z;c,0,c) = y |L((0,0) > (3ate, n) | 3y+¢ > 2x > 3y)| 2” 
n=c (mod 2) 
(zc) 


< o2(z;c+1)— O(zc)b(z3e+2)’ 8) 


with 
“n\= _1) (a—3!)/2 I 
oy steams "( l )e 


3l<a 
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10.6 Simple paths with a piecewise linear boundary 


In this section we generalize the one-sided linear boundary results in Corollary 10.3.2 
and Theorems 10.4.5 and 10.4.7 to piecewise linear boundaries. To be more precise, 
we want to count lattice paths from the origin (0,0) to (c,d) staying weakly below 
the line segments 


{(x,y):x=why+vV1,0=yo <y<yi}, {(y) x= boy t+ Va,y1 <y < yo}, 
wey {(%,Y) 2X = WY + Vins Ym—1 <Y S Ym =A}, (10.39) 


for some sequence 0 = yp < yj <--- < Ym =d of non-negative integers, non-negative 
integers 1, L2,...,[4m, and integers V), V2,..., Vm. Let us denote this piecewise linear 
restriction by Ry». See Figure 10.9 for an example. By an iteration argument it will 
be seen that the solution to this problem can be given in form of an m-fold sum. 

The result below is due to Niederhausen [98, Sec. 2.2 in connection with (2.4) and 
(2.7)], but see also [96]. In order to understand the statement below, it is important 
to observe that the number of paths that we want to determine is a polynomial in c, 
while keeping all other variables fixed. We shall not provide a detailed argument here 
but, instead, refer to [98, Sec. 2.2]. For convenience, let us denote this polynomial by 
LR», (C)- 


Theorem 10.6.1 The number of lattice paths from (0,0) to (c,d) staying weakly 
below the piecewise linear boundary Rm given in (10.39) is equal to 


Ym-1 
|L((0,0) — (c,d) | Rm)| = Y Lr, 1,i(Mmi + Vm — 1) 
i=0 


C— Lnd-Vn +1 ctd—i(umt+1)—Vn+1 
c+d—i(Umt+1)—V,+1 d-i 


), (10.40) 


Remark 10.6.2 Clearly, we may now apply Theorem 10.6.1 to Lr, i(fmit Vin — 1) 
so that, iteratively, we obtain an m-fold sum. (In the last step, one applies (10.22).) 


Idea of proof of Theorem 10.6.1. To begin with, let us assume that the piecewise 
linear boundary be convex. See Figure 10.9 for an example. Evidently, any path from 
(0,0) to (c,d) has to touch x = Umy + Vm for the last time, say in (Umi+ Vn, i). 
In Figure 10.9 we have m = 3, and the last touching point of the path P with x = 
Hny + Vm is the point (10,2), which is marked by a star. Then we utilize the same 
idea that led to the formula (10.21) to obtain the number in question being equal to 


|Z((0,0) + (c,d) | Rn) | 
Ym-1 
= YF |E((0,0) (Hint + Yn = 148) | Rn) 


. |L( (Mi + Vn, i) = (c,d) | x = Ly + Vm) | 
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(c,d) 


Figure 10.9 
A piecewise linear boundary. 


Ym-1 
= y |L((0,0) > (Limi + Vm — 1,i) | Rm—1)| 
i=0 


C—Umd—Vn +1 ctd—i(umt+1)—Vn+1 
c+d—i(fm +1)—Vn+1 d-i : 


by virtue of Theorem 10.4.5. In the summand, we were allowed to replace R,, by 
R»m_1 since the summation ends at i = y,,_1, and thus the mth segment does not come 
into play. 

Evidently, if the piecewise linear boundary should not be convex, then this argu- 
ment breaks down. However, Niederhausen shows in [98, Sec. 2.2], using the polyno- 
miality of the path numbers (and some results from umbral calculus), that the above 
formula continues to hold in that case also, even if the substitution of j,i + V,_,— 1 in 
the argument of the polynomial Lr,,_, ;(.) has no combinatorial meaning anymore. lf 


10.7 Simple paths with general boundaries 


The most general problem to encounter is to count paths in a region that is bounded 
by nonlinear upper and lower boundaries as exemplified in Figure 10.10. 

To have a convenient notation, let aq} < a2 <--- <a, and bj <bo <--- <b, be 
integers with a; > b;. We abbreviate a = (a1,d2,...,d,) and b = (bj, b2,...,b,). By 
L((0,b1) — (n,an) |a > y > b) we denote the set of all lattice paths from (0,51) 
to (n,a,) that satisfy the property that for all i = 1,2,...,n the height y; of the ith 
horizontal step is in the interval [b;,a;]. If we also write y(P) = (1, y2,-.--,Yn) for the 
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(0,b1) 


Figure 10.10 
A path in a ladder-shaped region. 


sequence of heights of horizontal steps of a path P, then the notation just introduced 
explains itself. Pictorially (see Figure 10.10), the described restriction means that we 
consider paths in a ladder-shaped region, the upper ladder being determined by a, the 
lower ladder being determined by b. See Figure 10.11, which displays an example 
with n = 6, a = (3,5,7,8,8,8), b = (0,1,1,2,5,5), y(Po) = (2,2,2,4,6,8). 

Originally, the result below was derived by Kreweras [85] using recurrence rela- 
tions, but the most conceptual and most elegant way to attack this problem is by the 
method of non-intersecting lattice paths; see Section 10.13 and [115]. 


Theorem 10.7.1 Let a = (a\,a2,...,dn) and b = (b1,b2,...,bn) be integer se- 
quences with ay <az <+++ <a, b) < bo < +++ < dy, and aj > b;, i=1,2,...,n. 
The number of all paths from (0,b1) to (n,an) satisfying the property that, for all 
i=1,2,...,n, the height of the ith horizontal step is between b; and a; is given by 


aj—bj+1 
(0.61) > (san) |a>y>b)|=, det, (( at )). (10.41) 


Proof. We apply Theorem 10.13.3 with A = (1,1,...,1) and p = (0,0,...,0), both 
vectors containing 7 entries. This counts vectors (7), 7,...,%) with t <™<-:-< 
7%, With a lower and an upper bound on each 7;. By replacing 7; by 7; — i, this counting 
problem is translated into the counting problem we consider here, 7; — i correspond- 
ing to the height of the ith horizontal step of a path. | 


Of course, with increasing n this formula will become less tractable. An alter- 
native formula can be obtained by rotating the whole picture by 90° and applying 
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Figure 10.11 
Lattice path with a general boundary. 


formula (10.41) to the new situation. Now the size of the determinant is a,, which is 
smaller than before if a, <n, i.e., if the difference between the y-coordinates of end 
and starting point is less than the difference between the respective x-coordinates. 

In some cases, a different type of formula might be preferable, which one may 
obtain by the so-called dummy path technique, as proposed in Krattenthaler and 
Mohanty [83]. Again, it comes from non-intersecting lattice paths. It is based on the 
following observation (see Stanley [111, Ex. 2.7.2]). 


Lemma 10.7.2 Let C,,C2,...,C, be pairwise distinct points in Z7. Then the number 
of lattice paths from (a,b) to (c,d) that avoid C,C2,...,Cy is given by 


det ( |L(A; — Ej) 


1<i,j<n+1 


i (10.42) 


where A, = (a,b), Aa =C\,..., Angt =Cy, E1 = (c,d), Fy = Ch, ..., Engi =Ch. 


Proof. We reformulate our counting problem in that we want to determine the num- 
ber of families (P,,P),...,P,+1) of non-intersecting lattice paths, where P; runs from 
A, = (a,b) to E; = (c,d), and for i = 1,2,...,n the “dummy path” P.,; runs from 
Aj+1 = C; to Ej, =C;. By Theorem 10.13.1, with G the directed graph with vertices 
Z? and edges given by horizontal and vertical unit steps in the positive direction, all 
weights being 1, A; and E; as above, this number equals the determinant in (10.42). 


The idea now is that, given some (possibly two-sided) boundary, one “describes” 
this boundary by such “dummy points” (paths) and uses the above lemma to compute 
the number of paths that avoid these, thus avoiding the boundary. In some cases the 
boundary can be “described” by only very few “dummy points,” which may lead to 
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Figure 10.12 
A Motzkin path. 


Figure 10.13 
A Schréder path. 


a useful formula. Several formulae that appear in the literature are instances of this 
idea (sometimes of minor variations), although it may not be stated there that way; 
see [95, Theorem 2 on p. 36] and [83] and the references given there. 


10.8 Elementary results on Motzkin and Schroder 
paths 


The subject of this section and the following three sections is lattice paths in Z7, 
which consist of up-steps (1,1), down-steps (1,—1), and level-steps (1,0) or (2,0) 
that do not pass below the x-axis. If the only allowed level-steps are unit steps (1,0), 
then the corresponding paths are called Motzkin paths. If the only allowed level- 
steps are double steps (2,0), then the corresponding paths are called Schréder 
paths. We call the special paths which consist of just up- and down-steps (but con- 
tain no level-steps) Catalan paths. In the special case, where these paths start and 
end on the x-axis, they are commonly called Dyck paths. 
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Figure 10.14 
A Catalan path. 


Figure 10.15 
A Schréder path rotated-reflected. 


Let M = {(1,1),(1,—1), (1,0)} and S = {(1,1), (1, —1), (2,0)}, so that ™ is the 
set of steps allowed in Motzkin paths (see Figure 10.12 for an example) and S is the 
set of steps allowed in Schréder paths (see Figure 10.13 for an example). 

A frequently used alternative way to view Schréder paths is by reflecting the 
picture with respect to the x-axis, rotating the result by 45°, and finally scaling ev- 
erything by a factor of 1/1/2, so that the steps (1,1),(1,—1),(2,0) are replaced by 
the steps (1,0), (0,1), (1,1), in that order. Figure 10.15 shows the result of this trans- 
lation when applied to the Schréder path in Figure 10.13. It translates Schréder paths 
into paths that consist of unit horizontal and vertical steps in the positive direction 
and of upward diagonal steps, which stay weakly below the main diagonal y = x. 
Without the diagonal restriction, the counting problem would be solved by the De- 
lannoy numbers in (10.6). 
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Nevertheless, this translation, combined with Theorem 10.3.1, already tells us 
how to enumerate Motzkin and Schréder paths with given starting and end point. 


Theorem 10.8.1 Let b > 0 and d > 0. The number of all paths from (a,b) to (c,d) 
that consist of steps out of M = {(1,1),(1,—1),(1,0)} and do not pass below the 
x-axis (Motzkin paths) is given by 


Ba) Cciae «wha! (ere ora) 
(10.43) 


where, by convention, a binomial coefficient is O if its bottom parameter is not an 
integer. 

Furthermore, the number of all paths from (a,b) to (c,d) that consist of steps out 
of S = {(1, 1), (1,—1), (2,0)} and do not pass below the x-axis (Schréder paths) is 
given by 


(e=a)/2 /._ 
|L((a,b) — (c,d);S|y>0)| = y ( : ‘) 


(ee seu wh) = (reece) ae) 


with the same convention for binomial coefficients. 


Proof. — By the above described translation (reflection + rotation), a Motzkin 
path from (a,b) to (c,d) with exactly k level-steps is translated into a path from 
(442, 45*) to (34, S4), which consists of steps from {(1,0), (0, 1), (5, 5)}, among 
them exactly k diagonal steps (5, 5), and which stays weakly below the main diag- 
onal y = x. Clearly, if we remove the k diagonal steps and concatenate the resulting 
path pieces, we obtain a simple path from (2, at) to (44 k, “4 x) which 
stays weakly below y = x. The number of the latter paths was determined in Theo- 
rem 10.3.1. On the other hand, there are (a) ways to reinsert the k diagonal steps. 
Thus, Eq. (10.43) is established. 

The proof of (10.44) is analogous. a 


We will derive expressions for corresponding generating functions in Sec- 
tion 10.9, see Theorem 10.9.2. 

It is worth stating the special case of Theorem 10.8.1 where the paths start and 
terminate on the x-axis separately. 


Corollary 10.8.2 The number of Motzkin paths from (0,0) to (n,0) is given by 


Wal (n\ 1 (2k 
|L((0,0) — (n,0);M | y > 0)| = py (3) ex): (10.45) 
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If n is even, the number of Schréder paths from (0,0) to (n,0) is given by 


2 Mni2+k\ 1 (2k 
2((0,0) + (n,0}:8| y>0)|= ¥ ( ss Jaa): (10.46) 


The numbers in (10.45) are called Motzkin numbers. The numbers in (10.46) 
are called large Schroder numbers. If n > 1, the latter are all divisible by 2 (which 
is easily seen by switching the first occurrence of a level-step with a pair consisting 
of an up-step and a down-step, and vice versa). Dividing these numbers by 2, we 
obtain the little Schréder numbers. Similarly to Catalan numbers, Motzkin and 
Schréder numbers also appear in numerous contexts; see [112, Ex. 6.38 and 6.39]. 

The summations in (10.43) and (10.44) do not simplify, not even in the special 
cases given in (10.45) and (10.46). 

In concluding this section, we point out that Motzkin paths, or, more precisely, 
decorated Motzkin paths, are of utmost importance for the enumeration of many 
other combinatorial objects, most importantly for the enumeration of permutations 
and (set) partitions. A decorated Motzkin path (in the french literature: “‘histoire’’) 
is a Motzkin path in which each step carries a certain label. In terms of enumera- 
tion, one may consider this as allowing several different steps of the same kind: for 
example, several different horizontal steps. In terms of generating functions, this la- 
beling is reflected by appropriate weights of the steps. The importance of decorated 
Motzkin paths comes from the fact that several bijections have been constructed be- 
tween them and permutations or partitions, which have the property that they “trans- 
fer” detailed information about permutations or partitions to the world of (decorated) 
Motzkin paths, allowing for very refined enumeration results for permutations and 
partitions. Such bijections have been constructed by Biane [9], Foata and Zeilberger 
[44], Francon and Viennot [46], Médicis and Viennot [94], and by Simion and Stan- 
ton [107]. See [33] for a unifying view. 


10.9 A continued fraction for the weighted counting of 
Motzkin paths 


We now assign a weight to each Motzkin path that starts and ends on the x-axis, and 
express the corresponding generating function in terms of a continued fraction. The 
corresponding result is due to Flajolet [42]. The weight is so general that the result 
also covers Schréder paths and Catalan paths. 

Given a Motzkin path P, we define the weight w(P) to be the product of the 
weights of all its steps, where the weight of an up-step is 1 (hence, does not contribute 
anything to the weight), the weight of a level-step at height h is by, and the weight 
of a down-step from height h to h — 1 is Ay. Figure 10.16 shows a Motzkin path with 
the steps labeled by their corresponding weights, so that the weight of the path is 
brAab bi Ashady = bib. AZ As. 


Then the following theorem is true. 
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Figure 10.16 
A labeled Motzkin path. 


Theorem 10.9.1 With the weight w defined as above, the generating function for 
Motzkin paths running from the origin back to the x-axis, which stay weakly below 
the line y =k, is given by 


GF (L((0,0) > (*,0);M |0<y<k);w) 


= . (10.47) 


nV 
[2h 


In particular, the generating function for all Motzkin paths running from the origin 
back to the x-axis is given by the infinite continued fraction 


GF (L((0,0) + (*,0);M|0<y);w) = : 7 . (10.48) 
if 


Ar 


oa = ere 


Proof. — Clearly, it suffices to prove (10.47). Equation (10.48) then follows upon 
letting k — 9, 

We prove (10.47) by induction on k. For k = 0, Equation (10.47) is trivially true. 
Hence, let us assume the truth of (10.47) for k replaced by k — 1. For accomplishing 
the induction step, we consider a Motzkin path starting at the origin, staying weakly 
below y = k, and finally returning to the x-axis, see Figure 10.17 for an example with 
k=3., 

Such a path can be uniquely decomposed into 


1uP\dluPod!?..., 


where / denotes a level-step at height 0, u an up-step, and d a down-step, where e; are 
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Figure 10.17 
The path with decomposition 1! uPd 7 uPyd [°uP3d. 


non-negative integers, and where, for any i, P; is some path between the lines y = 1 
and y = k that starts at and returns to the line y = 1. For example, this decomposition 
applied to the path in Figure 10.17 yields 


uP|dPuPodl°uP3d, 


where P; = uld, P, is the empty path, and P; = luudd. This implies immediately the 
generating function equation 


GF (L((0,0) > (*,0);M|0<y<k);w) 
1 
~ 1— by —A-GF(L((0,1) > (*,1);M|1<y <h);w) 


By induction, the generating function on the right-hand side is known: It is given by 
(10.47) with k replaced by k — 1, b; replaced by bj, and A; replaced by A;+1, for all 
i. This completes the induction step. | 


This result has numerous consequences. First of all, it allows us to derive alge- 
braic expressions for the generating functions )',59M,z" and ¥',>0Snz", where M,, 
denotes the number of all Motzkin paths from (0,0) to (1,0), and where S,, denotes 
the number of all Schréder paths from (0,0) to (2n,0). By definition, Mp = So = 1. 
The numbers M,, are called Motzkin numbers, while the numbers S, are called 
large Schroder numbers. 


Theorem 10.9.2 We have 


ee ree 
y Mz" = — 52 aes (10.49) 


n>0 


and 
haga ieere 
¥ Saz" = — x a (10.50) 


n>0 
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Proof. By (10.48) with b; = z and A; = 2 for all i (the reader should note that for 
any down-step there is a corresponding up-step before), we have 


Thus, in particular, we have M(z) = 1/(1—z—z°M(z)). The appropriate solution of 
this quadratic equation is exactly the right-hand side of (10.49). 
Similarly, by setting bj = A; = 2 in (10.48) for all i, we obtain 


1 
y Sav" = 2 ) 
n>0 1 5 z 
ss 2 
(eee 
oe fa een 
and eventually (10.50) after solving the analogous quadratic equation. | 


In Section 10.11, we will express the continued fraction (10.47) in numera- 
tor/denominator form, the numerator and denominator being orthogonal polynomi- 
als. 

We conclude this section with another continued fraction result, due to Roblet 
and Viennot [102]. We restrict our attention to Dyck paths, that is, to paths consisting 
of up- and down-steps, starting at the origin and returning to the x-axis, and never 
running below the x-axis. We refine the earlier defined weight w in the following way, 
so that in addition it also takes into account peaks: Given a Dyck path P, we define 
the weight #(P) of P to be the product of the weights of all its steps, where the weight 
of an up-step is 1, where the weight of a down-step from height h to h— 1, which 
follows immediately after an up-step (thus, together, forming a peak of the path), 
is V;, and where the weight of a down-step from height h to h— 1, which follows 
after another down-step, is A,,. Thus, the weight of the Dyck path in Figure 10.18 is 
V2 V4V4A3V3A2A4 V1 = V1 V2V3 VIA A2A3. With these definitions, the theorem of Roblet 
and Viennot [102, Prop. 1] reads as follows. 


Theorem 10.9.3 With the weight w defined as above, the generating function 
Ypw(P), where the sum is over all Dyck paths starting at the origin and returning to 
the x-axis, is given by 


GF (L((0,0) — (*,0);{(1, 1), (1,-1)} | y = 0);#) 


= . (10.51) 
1 (v1 A) 
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Figure 10.18 
A Dyck path. 


10.10 Lattice paths and orthogonal polynomials 


Orthogonal polynomials play an important role in many different subject areas, 
whether they be pure or applied. The reader is referred to [117] for an in-depth intro- 
duction. It is well-known that the theory of orthogonal polynomials is intimately con- 
nected with Hankel determinants and continued fractions, which we just discussed 
in Section 10.9 from a combinatorial point of view. It is Viennot [119] who made 
the connection, and who showed that a large part of the theory of orthogonal poly- 
nomials is in fact combinatorics. The key objects in this combinatorial theory of 
orthogonal polynomials are Motzkin paths. If one is forced to, one may compress 
the interplay between the theory of orthogonal polynomials and path enumeration to 
two key facts: First, (generalized) moments of orthogonal polynomials are generat- 
ing functions for Motzkin paths, see Theorem 10.10.3; second, generating functions 
for bounded Motzkin paths can be expressed in terms of orthogonal polynomials; see 
Theorem 10.11.1. But, of course, this combinatorial theory of orthogonal polynomi- 
als has much more to offer, of which we present an extract in this section, with a 
focus on path enumeration. 

We call a sequence (p,(x))n>9 of polynomials over C, where p,(x) is of degree n. 
orthogonal if there exists a linear functional L on polynomials over C (i.e., a linear 
map, which maps a polynomial to a complex number) such that 


0, ifn Am, 


nonzero, ifn=m. 


L(Pn(X)Pm(x)) = (10.52) 


We alert the reader that our definition deviates from the classical analytic definition in 
that we do not require L(pn(x)*) to be positive. The above somewhat weaker notion 
of orthogonality is sometimes referred to as formal orthogonality. The term “for- 
mal” expresses the fact that the corresponding theory does not require any analytic 
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tools, just formal, algebraic arguments. In fact, the formal theory could be equally 
well developed over any field K of characteristic 0 (instead of over C). 

It is easy to see that it is not true that for every linear functional L there is a cor- 
responding sequence of orthogonal polynomials. Let us consider the example of the 
linear functional defined by L(x") := 1,n =0,1,2,.... Equivalently, this means that 
L(p(x)) = p(1). In order to construct a corresponding sequence of orthogonal poly- 
nomials, we start with po(x). This must be a polynomial of degree 0, but otherwise we 
are completely free. Without loss of generality we may choose po(x) = 1. To deter- 
mine p(x), we use (10.52) with m= 0 and n = 1. Thus we obtain pi (x) =x—1. But 
then we have L(p1(x)*) = L((x—1)?) = 0, which violates the requirement (10.52), 
with m = n= 1, that L(p;(x)”) should be nonzero. 

On the other hand, if we have a linear functional L such that there exists a se- 
quence of orthogonal polynomials, then it is easy to see that all other sequences are 
just linear multiples of the former sequence. 


Lemma 10.10.1 Let L be a linear functional on polynomials and (py(x))n>0 be a 
sequence of polynomials orthogonal with respect to L. If (qn(x))n>0 is another se- 
quence of polynomials orthogonal with respect to L, then there are nonzero numbers 
an € C such that qn(x) = an pn(x). 


Lemma 10.10.1 justifies that from now on we will restrict our attention to se- 
quences of monic polynomials. 

One of the key results in the theory of orthogonal polynomials is Favard’s The- 
orem, which we state next. 


Theorem 10.10.2 A sequence (pn(x))n>0 of monic polynomials, p,(x) being of de- 
gree n, is orthogonal if and only if there exist sequences (bn)n>o and (An)n>1, with 
An 4 0 for all n > 1, such that the three-term recurrence 


XPn(X) = Pnsi(x) +bnPa(x) +AnPn—-i(x), — forn>1, (10.53) 
holds, with initial conditions po(x) = 1 and p\(x) =x— bo. 


In our context, more important than the statement of the theorem itself is its proof, 
which introduces Motzkin paths in a surprising way in (10.57), and in particular 
Theorem 10.10.3 below, which is the key ingredient in the proof, given after the 
proof of Theorem 10.10.3. 


Theorem 10.10.3 Let the polynomials p,(x) be given by the three-term recurrence 
(10.53), and let L be the linear functional defined by L(1) = 1 and L(pp(x)) =0 for 
n> 1. Then 


L(x" pe (x) pi(x)) = Any GF (L((0,k) > (n,1);M|O<y)sw), (10.54) 


where w is the weight on Motzkin paths defined in Section 10.9. 
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Proof. We prove the assertion by induction on n. 
If n = 0, then we have to show 


L(pe(x) pi(x)) = Ar Ar ber, (10.55) 


where OK denotes the Kronecker delta. We establish this claim by induction on k+/. 
It is obviously true for k = 1 = 0. Without loss of generality, we assume k > /. Then, 
using the three-term recurrence (10.53) twice, together with the induction hypothesis, 
we have 


L(px(x) pi(x)) = L(pe( nad 1 (x) — by-1L (pe (x) pra (x) — Ar-1L (pe (x) pi-a(x)) 
= L(xpe(x) pr-1(x)) 
ae x) pi-1(x)) + DL (pe (x) pi-1 (x) + AL (pe—1 (x) pi-1 (x) 
= Alpe ale ) pi-i(x))- 


Clearly, this achieves the induction step and thus establishes (10.55). 
We may now continue with the induction on n. For the induction step, we apply 
(10.53) with n = k on the left-hand side of (10.54). This leads to 


L(x" p(x) pi(x)) 
= L(x"! pesi(x) pila) + beL (2 pe (x) pi(x)) + AL (x! pei (x) pi(x)). 


By the induction hypothesis, we may interpret the right-hand side of this equality as 
a generating function for Motzkin paths, as described by (10.54) with n replaced by 
n—1. It is then straightforward to see that this implies (10.54) itself. a 


Now we have all the prerequisites available in order to prove Theorem 10.10.2. 


Proof of Theorem 10.10.2.__ For showing the forward implication, let (pp(x))n>0 
be a sequence of monic polynomials, p,(x) of degree n, which is orthogonal with 
respect to the linear functional L. Then we can express xp,(x) in terms of a linear 
combination of the polynomials py+1(x), Pn(x), ---, Po(x), 


XPn(X) = Pnti(X) + BnPa(x) + AnPn-1 (X) + On n—2Pn—2(x) + +++ + On opo(x)- 
(10.56) 
We have to show that in fact the first three terms on the right-hand side suffice, 1.e., 
that all other terms are zero. 

In order to do that, we multiply both sides of (10.56) by p;(x), for some i <<n—1, 
and apply L on both sides. Because of (10.52), on the right-hand side it is only the 
term @,,jL(p;(x)”) that survives. On the left-hand side we obtain L(xp;(x)pn(x)). 
The polynomial xp;(x) of degree i+ 1 can be expressed as a linear combination of 
the polynomials pj+1(x), pi(x), ..., po(x). Because of (10.52) and i<n—1, we 
therefore conclude that L(xpi(x) pn (x)) = 0. Hence, @,; is indeed 0 fori <<n—1. 
Similarly, we have 


AnL( Pa=1 (x)’) oa L(xpa (x) Pn (x)) = L(Pn (x)’) ) 


which is nonzero because of (10.52). Hence, we have A, 4 0, as desired. 
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For the proof of the backward implication, we must construct a linear functional 
L such that (10.52) holds, given a sequence (p,(x)) of polynomials, p,(x) of degree 
n, satisfying the three-term recurrence (10.53). We construct L by defining L(1) = 1 
and L(pn(x)) =0 for n > 1. Theorem 10.10.3 with n = 0 immediately implies that 
L(pe(x)pi(x)) = 0 if k A 1, as there is no Motzkin path from (0,k) to (0,/), and that 
L(pe(x)*) = A1-+- Ay A 0. This completes the proof of the theorem. | 


In the above proof, we have found a linear functional L by defining (see Theo- 
rem 10.10.3) its moments HU, := L(x") to be generating functions for Motzkin paths, 
namely 


Ln = GF (L((0,0) — (n,0);M|0<y);w) = yy w(P), 0.57) 
P a Motzkin path 
from (0,0) to (7,0) 


the weights of the paths carrying the coefficients in the three-term recurrence (10.53). 
This definition generates a linear functional with fo = L(1) = 1. It is easy to see 
that all other such linear functionals are constant nonzero multiples of the linear 
functional defined by (10.57). This justifies to restrict ourselves to linear functionals 
with first moment equal to 1. 

In view of Theorem 10.9.1, the backward implication of Theorem 10.10.2 can 
also be phrased in the following way. 


Corollary 10.10.4 Let (pn(x))n>0 be a sequence of polynomials satisfying the three- 
term recurrence (10.53) with initial conditions po(x) = 1 and pi(x) =x — bo. Then 
(Pn(x))n>0 is orthogonal with respect to the linear functional L, where the generating 
function of its moments Ll, = L(x") is given by 


y Un a= 
a 2 
n20 1—boz e 


(10.58) 


1—b 
ee Te bop 
All other linear functionals with respect to which the sequence (pn(x))n>0 is orthog- 
onal are constant nonzero multiples of L. 


Remark 10.10.5 A continued fraction of the type (10.58) is called a Jacobi contin- 
ued fraction or J-fraction. 


Proof of Corollary 10.10.4. | Combine (10.57) and (10.48) with b; replaced by b;z 
and A; replaced by (jz. | 


Below, we illustrate what we have found so far by an example. The polynomials 
which appear in this example, the Chebyshev polynomials, are of particular impor- 
tance for path counting. 
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Example 10.10.6 We choose b; =0 and A; = 1 for all i. Then the three-term recur- 
rence (10.53) becomes 


XUn(X) = Ungi (x) + Uni (x), forn>1, (10.59) 


with initial values uo(x) = 1 and u,(x) =x. These polynomials are, up to repara- 
metrization, Chebyshev polynomials of the second kind. 7o see that, recall that the 
latter are defined by 
sin((n+1)#) 
U, (cos 3) = Sao ; 
or, equivalently, 
sin((n + 1) arccos.x) 


Unt) Jaa 


Because of the easily verified fact that 
sin((n+1)0) +sin((n— 1)0) = 2cos 0 sinnd, 

the Chebyshev polynomials of the second kind satisfy the three-term recurrence 
2xU p(x) = Unsi(x) + Un-1(x), forn>1, (10.60) 


with initial values Uo(x) = 1 and U;(x) = 2x. Therefore we have 


Un (x) = un (2x) (10.61) 
for all n. 
It is straightforward to verify 
= kf k n—2k 
Un(x) =) (-1) arr (10.62) 
k>0 k 


whence, by (10.61), we have 


uals) = D-ne(" 


Another well-known fact is 


2. ff 1, n=m 
= i +1)8)si +1)8)dB =<? : 
=| sin((n + 1)8) sin((m +1)9) {0 ne 
Substitution of x = cos & then yields 
2 1 
- i; UU V1? a= Bix (10.63) 
= 


Thus the linear functional L for Chebyshev polynomials of the second kind is given 


by 
Lp(a) == [" oe) V1 Par 
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Using (10.57) we can now easily compute the corresponding moments. On the 
right-hand side of (10.57) all the terms corresponding to paths that contain a level- 
step vanish, because b; = 0 for all i. Therefore, the right-hand side counts paths that 
contain only up-steps and down-steps (and never pass below the x-axis). Clearly, 
there cannot be such a path if n is odd. If n is even, then by (10.11) the number of 
these paths is the Catalan number at (nyo): Hence, by also taking into account 
(10.61), we have shown that 


2 Pp ia [aan m=2n, 
MT J-1 


0, m=2n+1. 


Chebyshev polynomials are not only tied to Catalan paths (Dyck paths), i.e., 
paths that consist of just up- and down-steps, but also to Motzkin paths. To see this, 
let us now choose bj = A; = 1 for all i. Then the three-term recurrence (10.53) be- 
comes 

xMy (x) = my41 (x) +1 (x) + 1_1(x), forn>1, (10.64) 


with initial values mo(x) = 1 and m,(x) =x —1. Comparison with (10.60) reveals 
that these polynomials are expressible by means of Chebyshev polynomials of the 
second kind as 


mn(x) = Up () (10.65) 


We will take advantage of this relation in Section 10.11 to obtain further enumerative 
results on Motzkin paths. 


We now come back to the earlier observed fact that not all linear functionals allow 
for a corresponding sequence of orthogonal polynomials. Which linear functionals 
do is told by the following theorem. The criterion is given in terms of Hankel deter- 
minants of the moments of L. A Hankel determinant (or persymmetric or Turanian 
determinant) is a determinant of a matrix that has constant entries along antidiag- 
onals, i.e., it is a determinant of the form det) <;, i<n(di+ ay We omit the proof here, 
but Viennot [119, Ch. IV, Cor. 6 and 7] has shown that it can be given by an elegant 
application of the main theorem on non-intersecting lattice paths, Theorem 10.13.1, 
by using the interpretation of moments in terms of generating functions for Motzkin 
paths as given in Theorem 10.10.3. 


Theorem 10.10.7 Let L be a linear functional on polynomials with nth moment Ly = 
L(x"). For any non-negative integer n let 


Ho Mi be Ln 
Mi bo ....... Mn+1 
A, = det We a diocie se et Mn42 
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and 
Uo Uy tee Un-1 Un 
Uy U2 tee Un Un+1 
Xn = det : : : : 
Un-1 Un see Han—-2 U2n-1 
Un+1 Un42 --- Man Monti 


Let (pn(x))n>0 be the sequence of monic polynomials that is orthogonal with respect 
to L. Then the polynomials satisfy the three-term recurrence (10.53) with 


AnAn— 
Ay = (10.66) 
Ai-1 
and 
Xn Xn-1 
b,= 2 - ft. 10.67 
oh ae Weal (10.67) 


In particular, given a linear functional L on the set of polynomials, there exists a 
sequence of orthogonal polynomials that are orthogonal with respect to L if and only 
if all Hankel determinants A, = deto<i,j<n (Ui+;) of moments are nonzero. 


Implicit in (10.66) is the Hankel determinant evaluation 
Ke AG fee (10.68) 


which expresses the close interplay between Hankel determinants, moments of or- 
thogonal polynomials, and Motzkin path enumeration (via Theorem 10.10.3). 

We conclude this section with an explicit, determinantal formula for orthogo- 
nal polynomials, given the moments of the orthogonality functional. Again, Viennot 
[119, Ch. IV, 84] has given a beautiful combinatorial proof for this formula. using 
non-intersecting lattice paths. 


Theorem 10.10.8 Let L be a linear functional defined on polynomials with moments 
Mn = L(x"). Then the corresponding sequence (pn(x))n>0 of monic orthogonal poly- 
nomials is given by 


Ho ki be ves Un 
MW M2 ee Hn Mn+ 
Puls) =< P| ea a Oe | (10.69) 
ee ee cc 
Un-1 Ens Un4+1 oo U2n-1 
1 x baat 1 agltd x 


where, again, Ay—1 = deto<i,j<n—1(Mi+)): 
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Proof. It suffices to check that L(x” p,(x)) = 0 for 0 < m <n. Indeed, by (10.69) 
we have 


Ho My 2 vee Un 

Uy U2 Un Un+1 

L(x" Dn (x)) = det U2 Un Un+1 Un+2 

n—1 2 oe 

Mn-1 Un Mn+ 1 tee H2n-1 

Un Un+1 ae UMn+n-1 = Umtn 
Thus the result is zero, because for 0 < m <n the mth and the last row in the above 
determinant are identical. | | 


In Section 10.11 we derive several further enumeration results on Motzkin paths 
that feature orthogonal polynomials. 

We close this section by pointing out that Motzkin paths can be seen as so-called 
heaps of pieces. The corresponding theory has been developed by Viennot [120]. 
As a matter of fact, it is the combinatorial realization of the Cartier-Foata monoid 
[26]. 

For further intriguing work on the connections between lattice path counting, 
Hankel determinants, and continued fractions, the reader is referred to Gessel and 
Xin [52], and also Sulanke and Xin [116]. 


10.11 Motzkin paths in a strip 


In Sections 10.8 and 10.9 we have derived enumeration results for Motzkin paths 
that start and terminate on the x-axis. In particular, Theorem 10.9.1 provided a con- 
tinued fraction for the generating function with respect to a very general weight. This 
continued fraction can be compactly brought in numerator/denominator form, using 
orthogonal polynomials. In fact, more generally, a compact expression for the gen- 
erating function of Motzkin paths that start and terminate at arbitrary points can be 
given, again using orthogonal polynomials. 

In order to be able to state the corresponding result, we need two definitions. 
Recall that, given sequences (bn)n>0 and (An)n>1, with A, 4 0 for all n > 1, the 
three-term recurrence (10.53), 


XPn(x) = pn+i(x) + bnpa(x) + Anpn_1(), forn> 1, (10.70) 


with initial conditions po(x) = 1 and pi(x) =x — bo, produces a sequence (pp (x))n>0 
of orthogonal polynomials. We also need associated “shifted” polynomials (often 
simply called associated orthogonal polynomials), denoted by (Spy(x))n>0, which 
arise from the sequence (p,,(x)) by replacing A; by Aj, and b; by bj41,i=0,1,2,..., 
everywhere in the three-term recurrence (10.70) and in the initial conditions. Further- 
more, given a polynomial p(x) of degree n, we denote the corresponding reciprocal 


polynomial x” p(1/x) by p*(x). 
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Theorem 10.11.1 With the weight w defined as before Theorem 10.9.1, the gener- 
ating function for Motzkin paths running from height r to height s that stay weakly 
below the line y =k is given by 


y GF (L((0,r) > (n,s);M |0<y <k);w)x" 


n>0 
S—P j4* get * 
x Pr(a) Ball ifr<s, 
Pei t (10.71) 
x5 n* (x grt * (x . 
es Chas Care 


Phat (x) 
In particular, the generating function for Motzkin paths running from the origin back 


to the x-axis that stay weakly below the line y =k is given by 


YGF (L((0,0) > (n,0):M |0<y <k);w)x" = SPEC 
va Pia @) 


(10.72) 


Proof. Consider the directed graph, P41 say, with vertices vo,v1,...vx%, where for 
h=0,1,...,k—1 there is an arc from va, to v_41 as well as an arc from vpj+1 to va, 
and where there is a loop for each vertex v,. Motzkin paths that never exceed height 
k correspond in a one-to-one fashion to walks on P,,;. In this correspondence, an 
up-step from height / to h+ 1 in the Motzkin path corresponds to a step from vertex 
Vp, to vertex Vj, in the walk, and similarly for level- and down-steps. To make the 
correspondence also weight-preserving, we attach a weight of | to an arc from vy, to 
Vai1,4=0,1,...,k—1, a weight of A; to an arc from vj, to v__;, and a weight of by, 
to a loop at vp. 

By the transfer matrix method (see e.g. [111, Theorem 4.7.2]), the generating 
function for walks from v, to vs is given by 


(—1)'*8 det(I —xA;5,r) 
det(I — xA) 


where A is the (weighted) adjacency matrix of P,;, where J is the (k+1) x (k+1) 
identity matrix, and where det(J — xA;s,r) is the minor of (J —xA) with the sth row 
and rth column deleted. 

Now, the (weighted) adjacency matrix of P,,; with the property that the weight 
of a particular walk would correspond to the weight w of the corresponding Motzkin 
path is the tridiagonal matrix 


Fen He tO oy, 
A, by 1 0 Sag 
0 ra bh 1 0 
Wea, “Ee Sis i, Wa ee, =2 
O Agra Dg_r 1 0 
O Agi Dei 1 
0 An de 
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It is easily verified that, with this choice of A, we have det(J — xA) = pj, ,(x) (by 
expanding the determinant with respect to the last row and comparing with the three- 
term recurrence (10.53)), and, similarly, that the numerator in (10.71) agrees with 
(-1)'*5 det(I —xA;r,5). 


Example 10.11.2 We illustrate Theorem 10.11.1 for the special cases that were con- 
sidered in Example 10.10.6. 

Let first b; =0 and A; = 1 for all i. Combinatorially, we are talking about paths 
consisting of up- and down-steps, that is, Catalan paths (Dyck paths). Since for this 
choice of b;’s and A;’s there is no difference between the orthogonal polynomials and 


the corresponding associated orthogonal polynomials arising from (10.53), Exam- 
ple 10.10.6 tells us that 


Pn(x) = Spn(x) = Un(x/2). 
From (10.71), it then follows that 


LNE(Or) + @s)s{1),—D} 10 sy sh] 

. U,(1/2x)Up_s(1/2x) 
xU x41 (1/2x) moe (10.73) 

U;(1/2x) Up, (1 /2x) PE Ss . 
XUg4 1 (1/22) : a 


Next let bj = A; = 1 for all i. Combinatorially, we are talking about paths con- 
sisting of up-, down-, and level-steps, that is, Motzkin paths. Again, since for this 
choice of b;’s and A;’s there is no difference between the orthogonal polynomials and 
the corresponding associated orthogonal polynomials arising from (10.53), Exam- 
ple 10.10.6 tells us that 


Pal) = Spulx) = Un (5 *) | 


From (10.71), it then follows that 


y |L((0,r) > (n,s);M|0<y< k)| +x" 


n>0 
UKE) UB) pre, 
_ xU ps1 (7H) 40.74) 
1l—x 1—x : 
Us (Ci) Ur Vz) fi Ss. 
ee) 


Example 10.11.3 The standard application of (10.74) concerns the gambler’s ruin 
problem (see also [38, Ch. XIV]): two players A and B have initially a and R—a 
dollars, respectively. They play several rounds, in each of which the probability that 
player A wins is pa, the probability that player B wins is pg, and the probability 
that there is a tie is pr = 1— pa — pp. If one player wins, (s)he takes a dollar from 
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Figure 10.19 
The gambler’s ruin. 


the other. If there is a tie, nothing happens. The play stops when one of the play- 
ers is bankrupt. What is the probability that player A, say, goes bankrupt after N 
rounds? 

By disregarding the last round (which is necessarily a round in which B wins), 
this problem can be represented by a lattice path starting at (0,a—1), ending at 
(N —1,0), with steps (1,1) (corresponding to player A to win a round), (1,—1) 
(corresponding to player B to win a round), and (1,0) (corresponding to a tie), which 
does not pass below the x-axis, and which does not pass above the horizontal line 
y =R-—2. For example, the lattice path in Figure 10.19 corresponds to the play, 
where player A starts with two dollars, player B starts with four dollars, the outcome 
of the rounds is in turn TATBTTAABBBB (the letter A symbolizing a round where A 
won, with an analogous meaning of the letter B, and the letter T symbolizing a tie), 
so that A goes bankrupt after N = 12 rounds (while B did not). 

If we assign the weight pa to an up-step (1,1), pg to a down-step (1,—1), and 
pr toa level-step (1,0), then the probability of this play is the product of the weights 
of all the steps of the path P times pp (corresponding to the last round where B wins 
and A goes bankrupt; in our example, it is pr PAPT PBPTPTPAPAPBPBPBPB). If we 
write p(P) for the product of the weights of the steps of P, then, in order to solve 
the problem, we need to compute the sum Vp ppp(P), where the sum is over all the 
above described paths from (0,a— 1) to (N— 1,0). 

Clearly, (10.74) with r= a—1 and s = 0 provides the solution for the above 
problem, in terms of a generating function. Since the zeroes of the Chebyshev poly- 
nomials are explicitly known, one can apply partial fraction decomposition to obtain 
an explicit formula for the coefficients in the generating function. If this is carried 
out, then we get 


|L((0,r) > (n,s);M|0<y< k)| 


2 Tj "  aj(r+1) . aj(s+1) 
=——) 2 1} -<si Si . (10.7 
i ( cos a + ) sin a) sin kpD (10.75) 
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10.12 Further results for lattice paths in the plane 


In this section we collect various further results on the enumeration of two- 
dimensional lattice paths, respectively pointers to further such results. 

The first set of results that we describe concerns lattice paths in the plane integer 
lattice Z? that consist of steps from a finite set S that contains steps of the form (1,). 
Here, b is some integer. Say, 


S = {(1,b1), (1,b2),...,(1,Bm)}- (10.76) 


We also assume that to each step (1,b;) there is associated a weight w; € C. 
Banderier and Flajolet [5] completely solved the exact and asymptotic enumera- 
tion of lattice paths consisting of steps from S obeying certain restrictions. We con- 
centrate here on the exact enumeration results. 
The key object in their theory is the characteristic polynomial of the step set S, 


Ps(u) = Yo wyu?i. (10.77) 
j=l 
If we write c = —min;b; and d = max; b;, then Ps(u) can be rewritten in the form 


d 
Ps(u) = 2 pju, 


jac 


for appropriate coefficients p;. Associated with the characteristic polynomial is the 
characteristic equation 


1 — zPs(u) =0, (10.78) 
or, equivalently, 
c+d : 
uo — zu°Ps(u) =u —z) pj_cu! =0. (10.79) 
j=0 


The form (10.79) has only non-negative powers in u, and it shows that, counting mul- 
tiplicity, there are c-+d solutions to the characteristic equation when u is expressed 
as a function in z. These c+d solutions fall into two categories; There are c “small 
branches” uw; (z),u2(z),...,Uc(z) satisfying 


u;(z) nw e2tili- Die pl/er1/e asz— 0, 
and d “large branches” u.+1(Z),Uc+2(Z),---,Uc+a(z) Satisfying 


w e2ni(cti-J)/d 1/4 


pzl4g-Wa 


uj(z) as z—> 0. 


One can show that there are functions A(z) and B(z) that are analytic and non-zero at 
0 such that, in a neighborhood of 0, 


uj(z) = @F 12 '/¢A(@i}z!/¢), with o = e?*/", 7 =1,2,...,c, (10.80) 


uj(z) = oti Vap@ie 2/4) with@=e"/4,  j=ct+1,c+2,...,c+d. 
(10.81) 
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We are now in the position to state the enumeration results for lattice paths with 
steps from S without further restriction. In the formulation, we use ¢(P) to denote the 
length of a path P, and h(P) to denote the abscissa (height) of the end point of P. 


Theorem 10.12.1 The generating function Y pz‘?)u') for lattice paths P that start 
at the origin and consist of steps from S as given in (10.76) equals 


1 


aL (10.82) 


GF (L((0,0) > (*,*);S)32 uw") = 


with Ps(u) the characteristic polynomial of S given in (10.77). Moreover, the gener- 
ating function pz?) for those paths P that end at height 0 equals 


GF (L((0,0) + (+,0);8):2) =, 
j=l 


uj(Z) 


= ce (uy (z)u2(z) + uc(z)), (10.83) 


where uj (z),U2(z),..-,Uc(z) are the small branches given in (10.80). Finally, for k < 
c the generating function Vpzl?) for those paths P that end at height k equals 


GF (L((0,0) > (*,k);S)32“?) =) = (Late » (10.84) 


where again u;(z),u2(z),-.-;Ue(z) are the small branches given in (10.80), while for 
k > —d it equals 


GF (L((0,0) > (#,4):8);2") =-2 Yo w= 


j=ct+1 
(10.85) 
eo i elementary combinatorial principles, the generating function 
Yp (Py?) for lattice paths P that start at the origin and consist of steps from S 


is given - Yn>02" Pg (u), which equals (10.82). 

In order to determine the generating function Tez ’ for those lattice paths P 
that end at height 0, we have to extract the coefficient of uv° in (10.82). This can be 
achieved by computing the contour integral 


1 1 du 
10. 
eae u’ oe 


where C is a contour encircling the origin in the positive direction. One has to choose 
C so that, for sufficiently small z, the small branches lie within the contour, while 
the large branches lie outside. Then, by the residue theorem, only the small branches 
contribute to the integral (10.86). The residue at u = u;(z) equals (assuming that, in 
addition, we have chosen z so that all small branches are different) 


1 1 
re (a Pal 7) ~ aj(@)PECu)) 
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The integral in (10.86) equals the sum of these residues. This sum simplifies to 
(10.83) since differentiation of both sides of the characteristic equation (10.78) shows 
that P§(uj(z))~! = —2ul, (z) for all small branches u ;(z). 

The arguments for establishing (10.84) and (10.85) are similar. | | 


The second set of results concerns lattice paths starting at the origin with steps 
from S that do not run below the x-axis. 


Theorem 10.12.2 The generating function Y pz‘) u') for lattice paths P that start 
at the origin, consist of steps from S as given in (10.76), and do not run below the 
x-axis, equals 


GF (L((0,0) > (*,*):8 | y > 0) 32) = SEE 


1 c+d 1 
foe ea =. os) 
pak jad, uj) 


with Ps(u) the characteristic polynomial of S given in (10.77), and u;(z),u2(z),---, 
Uc(Z) and uc+1(Z),Uc42(Z),--+;Ucta(Z) the small and large branches given in (10.80) 


and (10.81). In particular, the generating function Yipee) for those paths P that end 
at height 0 equals 


Lele 
GF (L((0,0) > (*,0);S|y > 0);z) = Tse) 


(—1)7"! c+d 


1 
Pa J, uj(z)) (10.88) 


Proof. Here, we use the so-called kernel method (cf. e.g. [18]). Let F(z, u) denote 
the generating function on the left-hand side of (10.87). Then we have 


F(z,w) = 1+zPs(u)F(z,u) —z[u~°] (Ps(u)F(z,u)), (10.89) 


where [u<°]G(z,u) means that in the series G(z,u) all monomials zu” with m > 0 
are dropped, because any lattice path that is counted by F(z,u) is either empty, or 
it consists of a step (zPs(u) describes the possibilities) added to a path, except that 
the steps that would take the walk below level 0 are to be taken out (the operator 
[u<°] extracts the terms to be taken out). Since Ps(u) involves only a finite number 
of negative powers, we may rewrite (10.89) in the form 


F (z,u)(1 — zPs(u N=1-2F nt) (10.90) 


for some Laurent polynomials r;,(u) that can be computed from Ps(w) via (10.89), 


—k-1 


ry (ue) = [US°](Ps(u)u*) =)" pyul*. 
j=-c 


Here, F;(z) is the generating function )p z(P) for those paths P that end at height k. 
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In the current context, the factor | — zPs(u) on the left-hand side of (10.90) (which 
is identical with the left-hand side of the characteristic equation (10.78)) is called the 
kernel. The idea of the kernel method is to substitute u =u j(z), j=1,2,...,c (that is, 
the small branches) on both sides of (10.90) so that the kernel, and thus the left-hand 
side, vanishes. In this way, we arrive at the system of equations 


c-l 
uS(z)—z ¥uS(z)re(uj(z))Fe(z) =0, f= 1,2,...,€. 
k=0 


This system of linear equations in the unknowns F(z), Fi(z),...,Fce—1(z) could now 
be solved. Alternatively, we could observe that the expression 


c-l 
ue —Z y ur, (u) F(z) 
k=0 


is a polynomial in u of degree c with leading monomial u°. Its roots are exactly the 


small branches uj(z), j = 1,2,...,c. Hence, it factorizes as 
exd c 
uo —zZ ur, (u)Fi(2Z) = [](u-u,(z)). (10.91) 
k=0 j=l 


Extraction of the coefficient of u° on both sides immediately gives Fy(z), the gener- 
ating function for the paths that end at height 0. This leads directly to (10.88). The 
formula (10.87) follows from (10.90) and (10.91). | 


Sometimes, the kernel method is also applicable if the set of steps S is infinite. 
This is. for instance, the case for Lukasiewicz paths, which are paths consisting 
of steps from S; = {(1,b) >be {-1,0,1,2,... ae starting at the origin, returning 
to the x-axis, and never running below it. In that case, the equation (10.90) for the 
generating function Y pz!” Ju") becomes 


F(z,u) (1 aig)=! zu! Fo(z), (10.92) 


where, as before, F(z) is the generating function for those paths that end at height 0 
(that is, return to the x-axis). Here, the kernel is 
i 


en 


and it vanishes for u(z) = ie oe If this is substituted in (10.92), then we obtain 


1-V1-4 
22 , 
the Catalan number generating function (10.20). Hence, also Lukasiewicz paths of 
length n are enumerated by the Catalan number C,, = ; + a . 
To conclude this topic, it must be mentioned that Banderier and Gittenberger [6] 
have extended the analyses of [5] to also include the area statistics. 


GF (L((0,0) > (*,0);Sz | y >0);2) = F(z) = 
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A very cute problem, which arose in a probabilistic context around 2000, is the 
problem of counting paths (walks) in the slit plane. The slit plane is the integer 
lattice Z? where one has taken out the half-axis {(k,0) : k < 0}. Investigation of this 
problem started with the conjecture that the number of paths in the slit plane that 
start at (1,0) and do 2n + 1 horizontal or vertical unit steps (in the positive or in 
the negative direction) is given by the Catalan number C},;. This conjecture was 
proved by Bousquet-Mélou and Schaeffer in [20], but they provide much stronger 
and more general results on the enumeration of lattice paths in the slit plane in that 
paper. When it is not possible to find exact formulas, then the focus is on the nature of 
the generating function, whether it be algebraic or not, D-finite or not, etc. Methods 
used are the cycle lemma and the kernel method. 

An innocent looking three-candidate ballot problem stands at the beginning of 
another long line of investigation: Let E,,E2,E3 be candidates in an election, Fj 
receiving e; votes, Ey receiving e2 votes, and £3 receiving e3 votes, e; > max{e2,e3}. 
How many ways of counting the votes are there such that at any stage during the 
counting candidate EF) has at least as many votes as E> and at least as many votes as 
E37 In lattice path formulation this means to count all simple lattice paths in Z? from 
the origin to (e,,¢2,e3) staying in the region {(x1,x2,x3) : x, > x2 and x; > x3}. * 
We state the result below. Solutions were given by Kreweras [85] and Niederhausen 
[99], see also Gessel [48]. This line of research was picked up later by Bousquet- 
Meélou [16] who showed, again with the help of the kernel method, that the generating 
function of these “Kreweras walks” is algebraic. It must be pointed out that this 
counting problem is a “non-example” for the reflection principle (see Section 10.18), 
that is, the reflection principle does not apply. The reason is that, if one tries to set it 
up for application of the reflection principle, then one realizes that the nice property 
that for permutations other than the identity permutation some hyperplane has to be 
touched would fail. 


Theorem 10.12.3 Let e; > max{e2,e3}. The number of all lattice paths in Z? from 
(0,0,0) to (€1,€2,¢3) subject to x1 > x2 and x; > x3 is given by 


|L((0,0,0) — (€1,€2,¢3) | x1 > max{x7,x3})| 


= ci. €2 +63 Cee 
€1,€2,€3 T+e, \ €1,€2,€3 


SS (e; +e. +e3)! (28+ 27-2)! (i+ j—2)! 


ie Nes) jMe—pi@i-NlQj—-DiG+j+repr |? 


In particular, if e; = eg this number simplifies to 
|L((0,0,0) — (€1,€2,¢3) | x1 > max{x7,x3})| 


t  \? (2e;-2e3+1)! 
a eee : . (10.94 
(es (2e; +2)! we 


*It seems that this is a non-planar lattice path problem, contradicting the title of the section. However, the 
problem can be translated into a two-dimensional problem, see [16]. 
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We come to a relatively recent research field: the enumeration of walks in the 
quarter plane. The question that was posed is: Given a particular step set, can one 
find an explicit formula for the corresponding generating function, and, if not, is the 
generating function rational, algebraic, D-finite, or neither? For “small” step sets, 
the analysis is now complete, due to work by Bousquet-Mélou and Mishna [19], 
by Bostan and Kauers [12], and by Bostan, Kurkova, Raschel and Salvy [13, 14]. 
However, there is not yet a good understanding how, or whether at all, one can de- 
cide from the step set that the generating function has one of the above mentioned 
properties. 

The last topic that I mention here is the connection between Dyck and Schréder 
path enumeration on the one hand, and Hilbert series for diagonal harmonics and 
Macdonald polynomials on the other hand. This topic would by itself require a whole 
chapter. We refer the reader to the survey [61] and the references therein. One of the 
most intriguing combinatorial problems originating from the investigations in this 
area is new Statistics for Dyck paths, most prominently “bounce” and “dinv.” It has 
been shown (algebraically) that the pair (bounce. area) is equally distributed as (area, 
bounce), and the same for area and dinv. However, although much effort has been put 
into it, so far nobody could come up with a direct combinatorial reason (in the best 
case: a bijection) why this symmetry holds. 


10.13 Non-intersecting lattice paths 


The technique of non-intersecting lattice paths is a powerful counting method. We 
have already seen its effectiveness in Section 10.7. Originally, non-intersecting paths 
arose in matroid theory, in the work of Lindstrém [88]. Lindstrém’s result was redis- 
covered (not always in its most general form) in the 1980s at about the same time in 
three different communities, not knowing the work of each other at that time: in sta- 
tistical physics by Fisher [41, Sec. 5.3] in order to apply it to the analysis of vicious 
walkers as a model of wetting and melting, in combinatorial chemistry by John and 
Sachs [70] and Gronau, Just, Schade, Scheffler and Wojciechowski [58] in order to 
compute Pauling’s bond order in benzenoid hydrocarbon molecules, and in enumer- 
ative combinatorics by Gessel and Viennot [50, 51] in order to count tableaux and 
plane partitions. It must however be mentioned that in fact the same idea appeared 
even earlier in work by Karlin and McGregor [71, 72] in a probabilistic framework, as 
well as that the so-called “Slater determinant” in quantum mechanics (cf. [108] and 
[109, Ch. 11]) may qualify as an “ancestor” of the determinantal formula of Lind- 
str6m. Since then, many more applications have been found, particularly in plane 
partition and rhombus tiling enumeration, see e.g. [23, 30, 40, 114] and Chapter 9 
for more information on this topic. 

We devote this section to developing the theory of non-intersecting lattice paths 
and give some sample applications. This will be continued in Section 10.14, where 
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Figure 10.20 
A family of non-intersecting paths. 


we give results on the enumeration of non-intersecting lattice paths in the plane with 
respect to turns. 

The most general version of the non-intersecting path theorem ([88, Lemma 1], 
[51, Theorem 1]) is formulated for paths in a directed graph. Let G be a directed 
graph with vertices V and (directed) edges E. A path (actually, the usual notion in 
graph theory is walk) in G is a sequence vo,v1,...,Vm of vertices, for some m, such 
that there is an edge from 1; to v;4,;,i=0,1,...,m— 1. We denote the set of all paths 
in G from A to E by Lg(A — E). The directed graph G is called acyclic if there is no 
non-trivial closed path in G, i.e., if there is no path that starts and ends in the same 
vertex other than a zero-length path. 

The central definition is that a family P = (P,,P),...,P,) of paths P; in G is 
called non-intersecting if no two paths of P have a vertex in common. Otherwise P 
is called intersecting. In the context of lattice path enumeration, the graph G comes 
from a lattice. In many examples, the vertices of G are the lattice points Z? in the 
plane, and the edges of G connect a point (i, j) to (i+1,j), respectively a point 
(i,j) to (i,j +1). Figure 10.20 displays a family of non-intersecting lattice paths 
in this sense, Figure 10.21 shows a family of intersecting lattice paths. (It is very 
important to note that, in the geometric realization of paths as piecewise linear trails, 
the corresponding trails may very well have common points, but never in starting 
and end points of steps, see Figure 10.22 for such an example. In particular, non- 
intersecting lattice paths may even cross each other in the geometric visualization.) 

Returning to the general setup, we furthermore assume that to any edge e in 
the graph G there is assigned a weight w(e) (an element in some commutative ring 
A). The weight of a path P is the product w(P) = [].w(e), where the product is 
over all edges e of the path P. The weight w(P) of a family P = (P{,P2,...,P,) of 
paths is defined as the product of all the weights of paths in the family, w(P) = 
w((Pi,P2,---,Pr)) = TT, w(F). 
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Figure 10.21 
A family of intersecting paths. 


Figure 10.22 
Non-intersecting lattice paths may even cross. 
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Given two sequences A = (A1,Ap,...,A,) and E = (EF), £,...,E,) of vertices of 
G, we write Lg(A — E) for the set of all families (P;,P2,...,P,) of paths, where P, 
runs from A; to E;, i= 1,2,...,n, whereas Lg(A —> E | non-intersecting) denotes the 
subset of families of non-intersecting paths. 

We need one more piece of notation. Given a permutation o € G,, and a vector 
V = (V1,V2,---,¥n), by Vo we mean (Vg(1),Vq(2);-++Vo(n)). We are now in the posi- 
tion to state and prove the main theorem on non-intersecting paths, due to Lindstrém 
[88, Lemma 1]. 


Theorem 10.13.1 Let G be a directed, acyclic graph, and let A = (A,A2,...,;An) 
and K = (Ej,E),...,E,) be sequences of vertices in G. Then 


y (sgno) - GF (Lg(Ac — E | non-intersecting); w) 
o€G, 


= det (GF(Lc(Aj— E;);w)). (10.95) 


1<i,j<n 


Proof. By expanding the determinant on the right-hand side of (10.95), we obtain 
n 
ey (GF (L¢(4j; > Ei);w)) = p> sol ler (Lo(Agii) > Ei);) 
— y sgnow(P). (10.96) 
(o,P) 
o€G,,,PELgG(Ag—E) 


The sum in (10.96) expresses the determinant in (10.95) as a generating function for 
pairs (o,P) in the set 
LJ) Le(Ac > E). (10.97) 
ocGn 

We now define a sign-reversing, weight-preserving involution @ on the set of all 
pairs (o, P) in (10.97) with the property that P is intersecting. Sign-reversing means 
that if p((o,P)) = (O,Pg) then sgno = —sgn og, while weight-preserving means 
that w(P) = w(P»). Suppose that we had already constructed such a @. Then in the 
sum (10.96) all contributions of pairs (oP) in (10.97) where P is intersecting would 
cancel. Only contributions of pairs (o,P) in (10.97) where P is non-intersecting 
would survive, establishing (10.95). 

Next we construct the sign-reversing, weight-preserving involution @. Let (o,P) 
be in Lg(Ag — E) where P is intersecting. In the left-hand picture of Figure 10.23 
an example is shown with G the directed graph corresponding to the integer lattice 
Z?,n =3, and o = 213. Among all pairs of paths with a common point, choose the 
lexicographically largest, say (P;,P;), i< j, and among all common points of that 
pair choose the last along the paths. (It does not matter on which of the two paths 
of the pair we choose the last common point since the graph G is acyclic.) Denote 
this common point by M. In our example, the common points between paths are 
(3,2), (3,3), (4,5). The lexicographically largest pair of paths with common points 
is (P, P3). The last common point of this pair is M = (4,5). 
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o =213 o’ = 231 


Figure 10.23 
Lattice paths corresponding to 213 and 231. 


Returning to the general construction of the involution @, we now interchange 
the initial portions of P; and P; up to M. To be more precise, we form the new paths 


P; = (subpath of P; from Ag, ;) to M joined with subpath of P; from M to Ej] 
and 
P’ = [subpath of P; from Ag) to M joined with subpath of P; from M to Ej). 


Then we define 


((G;P)) =@((6, Pita Bist Pies Pe))) 
=(Go(E)), Pict P23 Peng Ba); 


where (i, /) denotes the transposition interchanging i and j. The right-hand picture 
in Figure 10.23 shows what is obtained by this operation in our example. The image 
((o,P)) is again an element of the set in (10.97) since the new permutation of 
the starting points of P is exactly oo (ij). Moreover, (Pi,...,P!,...,P),-.+;Pn) is 
intersecting since P’ and P’ are. From all this it is obvious that when @ is applied to 
((o,P)) we arrive back at (o,P). Hence, @ is an involution. Since o and 6 0 (ij) 
differ in sign, @ is sign-reversing. Finally, since the total (multi)set of edges in the 
path families does not change under application of @, the map @ is also weight- 
preserving. This finishes the proof. | 


The most frequent situation in which the general result in Theorem 10.13.1 is 
applied is the one where non-intersecting paths can only occur if the starting and 
end points are connected via the identity permutation, that is, if (P|, P2,...,P,) can 
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only be non-intersecting if P; connects A; with E;, i= 1,2,...,n. In that situation, 
Theorem 10.13.1 simplifies to the following result. 


Corollary 10.13.2 Let G be a directed, acyclic graph, and let A = (Aj,A2,..-,An) 
and KE = (E,,E»,...,En) be sequences of vertices in G such that the only permutation 
o that allows for a family (P,,P2,...,P,) of non-intersecting paths such that P; con- 
nects Agy;) with Ej, i= 1,2,...,n, is the identity permutation. Then the generating 
function Y-p w(P) for families P = (P1,P2,...,P) of non-intersecting paths, where P, 
is a path running from A; to Ej, i= 1,2,...,n, is given by 


GF (Lc(A > E | non-intersecting);w) = fet (GF (L¢(Aj > Ej);w)). (10.98) 
<i,jsn 


The standard application of Corollary 10.13.2 concerns semistandard tableaux. 
These are important objects particularly in the representation theory of the general 
and the special linear groups, cf. [103]. 

Let A = (A), A2,...,An) and = (1, Mo,..., Hn) be n-tuples of integers such that 


My > Ag > Ag >° > An, (10.99a) 
My = be 2 bg 2 2 Un, (10.99b) 
and A; > py; for all i. (10.99c) 


A semistandard tableau T of shape A / is an array of integers 


(10.100) 


TT es Tn An 


such that entries along rows are weakly increasing and entries along columns are 
strictly increasing. A semistandard tableau of shape (7,6,6,4)/(3,3,1,0) is shown 
in Figure 10.24(a). (The lower and upper bounds on the entries displayed to the left 
and right of the tableau should be ignored at this point.) 

Let a = (a1,a2...,d,) and b = (b1,b2,...,bn) be sequences of integers such that 


ay <a, <-+++< ay (10.101a) 
by < bo <-++ <p, (10.101b) 
and a; > b; for all i. (10.101c) 


We claim that semistandard tableaux of shape A/u where the entries in row i 
are at most a; and at least b; bijectively correspond to families (P,,P2,...,P,) of 
non-intersecting lattice paths P,, where P, runs from (UW; — i,b;) to (A; — i, a;). 

This is seen as follows. Let 2 be a semistandard tableau of shape A /u where 
the entries in row i are at most a; and at least b;. The semistandard tableau 7 is 
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—2< ll45 <7 
-1l< 5.5 7 <8 
O< 25 69 9 <9 
I< 1 5 6 8 < 10 


(a) Semistandard tableau 


(b) Family of non-intersecting lattice paths 


Figure 10.24 
A column-strict reverse plane partition and the corresponding family of non- 
intersecting lattice paths. 


mapped to a family of lattice paths by associating the ith row of 7 with a path P; from 
(4; — i,b;) to (A; —i,a;) where the entries in the ith row are interpreted as heights of 
the horizontal steps in the path P;. Thus from 7 we obtain the family P = (P,,...,P,) 
of lattice paths. Figure 10.24(b) displays the family of lattice paths that in this way 
results from the array displayed in Figure 10.24(a). 

Clearly, the property that the columns of 7 are strictly increasing translates into 
the condition that (P;,P),...,P,) is non-intersecting. 

By applying Corollary 10.13.2 to this situation, we obtain the following enumer- 
ation result. 


Theorem 10.13.3 Let A = (A1,A2,...,An) and MW = (1, L2,---;Hn) be sequences 
of integers satisfying (10.99). Let a = (a),d2,...,d,) and b = (b1,b2,...,bn) be 
sequences of integers satisfying (10.101). Then the number of all semistandard 
tableaux 1 of shape A/U where the entries in row i are at most a; and at least b; 
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ic Hee Fahy (10.102) 
1<i,j<n Ai—Mj-it+j 


equals 


More generally, the generating function gi” for the same set of semistandard 
tableaux 1, where n(1) denotes the sum of all entries of m, equals 


bap gy (Ae bpAr— Bp-tty 
st, (¢ ; | ffi ale (10.103) 


If the shape A / is a straight shape and the bounds a and b are constant, that is, 
if, say, LW = (0,0,...,0), b = (1,1,...,1), and a = (a,a,...,a), then the above de- 
terminants can be evaluated in closed form. Rewritten appropriately, the result is the 
hook-content formula. (We refer the reader to [103, Sec. 3.10] or [112, Cor. 7.21.6] 
for unexplained terminology). 


Theorem 10.13.4 Let A = (A,,A2,...,An) be a sequence of non-negative integers 
satisfying (10.99). Then the number of all semistandard tableaux 1m of shape 4 with 
entries in row i being positive integers at most a equals 


a+c(p) 
ae 10.104 
6) roe 


where the product is over all cells p in the Ferrers diagram of the partition 4, c(p) 
is the content of the cell p, and h(p) is the hook-length of the cell p. More generally, 
the generating function )\, gq’) for the same set of semistandard tableaux 1% equals 


» air gttelP) 
gui=! I] Tiley (10.105) 

By introducing more general weights, in the same way one can provide com- 
binatorial proofs for the Jacobi—Trudi-type identities for Schur functions (cf. [103, 
Sec. 4.5]), respectively formulas for so-called flagged Schur functions, originally in- 
troduced by Lascoux and Schiitzenberger [87], see also [121]. 

If in an array (10.100) one also introduces a relationship between the first row 
and the last row, then one is led to define so-called cylindric partitions, as was done 
by Gessel and Krattenthaler [49]. Also in that theory, non-intersecting paths play an 
essential role. 

It may seem that the general result in Theorem 10.13.1 is a very artificial state- 
ment, perhaps being of no use. However, even this result does have several applica- 
tions. For example, the most elegant proof of the determinant formula for higher- 
dimensional path counting under a general two-sided restriction ([115]; see Sec- 
tion 10.17, Theorem 10.17.1) fundamentally makes use of the full generality of The- 
orem 10.13.1. Further applications of the general formula (10.95) can be found in 
rhombus tiling enumeration (see [30, 40]), in combinatorial commutative algebra 
(see [79]), and in the combinatorial theory of orthogonal polynomials developed in 
Section 10.10 (see the proof of Theorem 10.10.8 as given in [119]). 
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Figure 10.25 
A perfect matching. 


In several applications, one has to deal with the problem of enumerating non- 
intersecting lattice paths where the starting and end points are not fixed. Either it is 
only the starting points that are fixed and the end points are any points from a given 
set (or the other way round), or it is even that the starting points may come from one 
set and the end points from another. The solution to these counting problems comes 
from Pfaffians. 

A Pfaffian is very similar to a determinant. Whereas in the definition of a deter- 
minant there appear permutations, in the definition of a Pfaffian there appear perfect 
matchings. A perfect matching of a set of objects, & say, is a pairing of the ob- 
jects. For example, if « = {1,2,3,4,5,6}, then {{1,3}, {2,5}, {4,6}} is a match- 
ing of &. A matching of {1,2,...,N} can be realized geometrically by drawing 
points labeled 1,2,...,N along a line, and then connecting any two points whose 
labels are paired in the matching by a curve, so that there are no touching points 
between curves and no triple intersections. Figure 10.25 shows the geometric real- 
ization of {{1,3}, {2,5}, {4,6}}. Any two pairs {i,k} and {j,/} in a matching for 
which i < j <k </ are called a crossing of the matching. The sign sgn7 of a 
matching z is (—1)°, where c is the number of crossings of 2. Thus, the sign of 
{{1,3}, {2,5}, {4,6}} is (—1)? = +1. In the geometric realization of a matching, its 
sign can be read off as (— 1°; where c’ is the number of crossing points between two 
curves. (It is easily checked that it does not matter how we draw the curves.) 

With these definitions, the Pfaffian Pf(A) of an upper triangular array A = 
(aij) 1<i<j<2n is defined by 


Pf(A) := y sent [J aij. (10.106) 
7 a perfect matching of {1,2,...,2n} {i,j}€n 


For example, for n = 2 we have 
PE ((dij)1<icj<4) = 412034 — 413424 + 414073. 


Alternatively, the Pfaffian could be defined as the (appropriate) square root of a 
skew symmetric matrix. To be precise, if A is a skew symmetric matrix, then 


Pf(A)* = det(A), (10.107) 


where Pf(A) has to be interpreted as the Pfaffian of the upper triangular part of A. For 
a (combinatorial) proof of this fact see e.g. [114, Prop. 2.2]. 
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Now let again G be a directed, acyclic graph. Let A,,Ao,...,A, be vertices in 
G, and E = (...,£1,£2,...) be an ordered set of vertices. What we want to count 
is the number of all families P = (P;,P),...,P,) of non-intersecting paths, where 
P; runs from A; to some vertex in E, i = 1,2,...,n. The difference to the situation 
in Theorem 10.13.1 is that the end vertices of the paths are not fixed. Neverthe- 
less, we adopt the earlier notation for this more general situation. To be precise, by 
Lg(A — E | non-intersecting) we mean the set of all families P = (P,,P),...,P,) 
of non-intersecting paths in G, where P; runs from A; to some vertex E;, in E, 
i=1,2,...,nandk; <ko <-::<k,. The corresponding enumeration result is due to 
Okada [100, Theorem 3] and Stembridge [114, Theorem 3.1]. 


Theorem 10.13.5 Let G be a directed, acyclic graph with a weight function w on its 
edges. Let A = (A1,A2,...,A2n) and E = (...,E£1,E2,...) be sequences of vertices in 
G. Then 


y (sgno)-GF (Lo(Ac >E| non-intersecting); ) = Phi <icj<o(Qeli, jsw)), 
CCG, 


(10.108) 
where Lg(Ag — E| non-intersecting) is the set of all families (P,,P2,...,P2n) of non- 
intersecting paths, P; connecting Agii) with Ey,, i=1,2,...,2nand kj <kz <...kan, 
and Qg(i, j;w) is the generating function ¥\p pr w(P')w(P") for all pairs (P',P") 
of non-intersecting lattice paths, where P' connects A; with some Ex and P" connects 
Aj with some Ej, with k <. 


The proof uses the same involution idea as the proof of Theorem 10.13.1 does. 
See [114, Proof of Theorem 3.1]. 

Similarly to Theorem 10.13.1, the most frequent situation in which the general 
result in Theorem 10.13.5 is applied is the one where non-intersecting paths can 
only occur if the starting and end points are connected via the identity permutation, 
that is, if (P,,P2,...,P2,) can only be non-intersecting if P; connects A; with Ex; 
i=1,2,...,2nand kj < kz <--- < ko». In that situation, Theorem 10.13.5 simplifies 
to the following result. 


Corollary 10.13.6 Let G be a directed, acyclic graph with a weight function w on its 
edges. Let A = (Aj,A2,...,A2) and E = (...,E,,E2,...) be sequences of vertices 
in G such that the only permutation © that allows for a family (P,,P2,...,P2,) of 
non-intersecting paths such that P; connects Agy;) with Ex, i= 1,2,...,2nand kj < 
ka <+++ < kag, is the identity permutation. Then the generating function Vp w(P) for 
families P = (P\,Po,...,P2n) of non-intersecting paths, where P, is a path running 
from A; to Ex, i= 1,2,...,2n and ky <ky < +++ < kon, is given by 


GF (L¢(A > E| non-intersecting);w) = Pfi<i<j<n(Qc(i,j;w)), (10.109) 
where Qg(i, j;w) has the same meaning as in Theorem 10.13.5. 


Theorem 10.13.5 and Corollary 10.13.6 are only formulated for an even number 
of paths. However, a simple trick allows us to also use it for an odd number of paths: 
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One introduces a “phantom” vertex X that cannot be reached by any other vertex (one 
can think of the vertex at infinity), and adjoins this point as a new starting point and 
as a new end point. A family of non-intersecting paths would necessarily contain the 
zero-length path from X to X as one of the paths, which therefore can be ignored. 
Theorem 10.13.5 applies, and yields the following corollary. 


Corollary 10.13.7 Let G be a directed, acyclic graph with a weight function w on its 
edges. Let A = (A1,A2,...,Aan—1) and E = (...,E\,E2,...) be sequences of vertices 
in G. Then 


y (sgno) - GF (Lg(Ag + E | non-intersecting);w) = Pf(Qg(i, j;w)) 1<i<j<2n, 
CCG, 
(10.110) 
where Lg(Aog — E | non-intersecting) has the same meaning as in Theorem 10.13.5, 
where for j <2n—1 the quantity Qg(i, j;w) has the same meaning as in Theo- 
rem 10.13.5, and where Qg(i,2n;w) is the generating function Y pw(P) for all paths 
running from A; to some point of E. 

In particular, if the vertices A and E are such that the only permutation © that 
allows for a family (P,,P2,...,P2n—1) of non-intersecting paths such that P; connects 
Ao(i) with Ex, t= 1,2,...,2n—1 and ky < kp <+++ < kon-1, is the identity permu- 
tation, then the generating function Y.pw(P) for families P = (P\,P2,..-,Pon-1) of 
non-intersecting paths, where P; is a path running from A; to Ex,, i=1,2,...,2n—1 
and ky < kz < +++ < kan-1, is given by 


GF (Lg(A > E| non-intersecting);w) = Pf(QG(i, j;w))1<icj<on. (10.111) 


For various applications of this theorem, see e.g. [77, 100, 114]. 

Next we consider a mixed case, in which the starting points of the paths are fixed, 
some end points are fixed, but some end points can be chosen from a given set. To 
be precise, let m and n be a positive integer with m <n, let A = (Aj,A2,...,An) 
and E = (E£),£,...,Em) be vertices in a given directed, acyclic graph G, let E = 
(...,£1,£2,...) be an ordered set of (finitely many or infinitely many) vertices. What 
we want to determine is the generating function for all families P = (P,,P),...,Pn) 
of non-intersecting paths, where for i=1,2,...,m the path P; runs from A; to E;, and 
where fori=m-+1,m+2,...,n the path P; runs from A; to some point Ex; in E, with 
king <Kma2 <+++ < kn. We write Lg(A —> EUE | non-intersecting) for the set of 
these families of paths. The corresponding enumeration result is due to Stembridge 
[114, Theorem 3.2]. 


Theorem 10.13.8 Let G be a directed, acyclic graph with a weight function w on its 
edges, and let m and n be a positive integer such that m <n and m+n is even. Let 
A = (A},A2,...,An), E= (E},E2,...,Em) and E = (...,£,,E5,...) be sequences of 
vertices in G. Then 


by (sgno)-GF (Lg(Ac + EUE | non-intersecting);w) = Pf (a a) ; 
o€G, 


(10.112) 
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where Lg(Ag —> EUE | non-intersecting) is the set of all families (P,,P2,...,Ph) 
of non-intersecting paths, P; connecting Agi;) with Ej, i= 1,2,...,m, P; connect- 
ing Agi, with Ex, i=m+1,m+2,...,n and kms < Kn42 < +++ < ky, where 
O = (Oc(i, j3w))1<i,j<n is a skew-symmetric matrix with Qg(i, j;w) denoting the 
generating function py pry w(P')w(P") for all pairs (P', P”) of non-intersecting lat- 
tice paths, where P' connects A; with some E, and P" connects Aj with some E, 
with k <1, and where H = (Hc(i, j;w))1<i<n,1<j<m ts the rectangular matrix with 
Hg(i, j;w) denoting the generating function ¥.pw(P) for all paths P from A; to Ej. 
The Pfaffian of a skew-symmetric matrix has to be interpreted according to the re- 
mark containing (10.107). 

In particular, if the vertices A and EUE are such that the only permutation o 
that allows for a family (P, P2,...,P,) of non-intersecting paths such that P; connects 
Agi) with Ej, i=1,2,...,m, and P, connects Agi; with Ex, i=m+1,m+2,...,n 
and kat <km42 <+++ < Ky, is the identity permutation, then the generating function 
Ypw(P) for all families (P|, P2,...,P,) of non-intersecting lattice paths, where for 
i=1,2,...,m the path P; runs from Ai to Ei, and ial as re =m+1,m+42,...,n the 
path P, dane, to some point Ex, ink, Kimi < kn42 <-+++ < ky, is given by 


GF (Lg(A + EUE | non-intersecting); w) = (-1)@) Pf (2 4) - (10.113) 
Again, the proof uses the same involution idea as the proof of Theorem 10.13.1 
does. See [114, Proof of Theorem 3.2]. Applications can for example be found in 
[31, 32, 114]. 
As a matter of fact, Theorem 10.13.8 is a special case of the so-called minor 
summation formula due to Ishikawa and Wakayama [66, Theorem 2]. 


Theorem 10.13.9 Let m, n, p be integers such that n+m is even and0 <n—m <p. 
Let M be any n x p matrix, H be any nx m matrix, and A = (ajj)1<i,j<p be any 
skew-symmetric matrix. Then we have 


LPE (AR) det (Mx : H) = (—1)(2) Pf (4 4 . 


where K runs over all (n—m)-element subsets of {1,p], A& is the skew-symmetric 
matrix obtained by picking the rows and columns indexed by K, and Mx is the sub- 
matrix of M consisting of the columns corresponding to K. 


Theorem 10.13.8 results from the special case of Theorem 10.13.9 where A is the 
P X p skew-symmetric matrix with all 1s above the diagonal, and M and H are ma- 
trices the entries of which are appropriately chosen path generating functions. This 
is based on the well-known fact (see e.g. [114, Prop. 2.3(c)]) that Pf(1)1<j<j<ow = 1 
for all NV. 

The last theorem in this section addresses the case where starting and end 
points are chosen from given sets. To be precise, let A = (Aj,Az,...,A,) and 
E = (...,£1,E2,...) be ordered sets of vertices (finitely many or infinitely many 
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in the case of E). What we want to determine is the generating function for all fam- 
ilies P = (P,,P2,...,Ps) of non-intersecting paths, where for i = 1,2,...,8 the path 
P, runs from some Ax, to some E;,. The corresponding enumeration result is due to 
Okada [100, Theorem 4] and Stembridge [114, Theorem 4.1]. In the formulation 
below, by abuse of notation, A’ C A means that A’ is a subsequence of A, with an 
analogous meaning for E’ C E. 


Theorem 10.13.10 Let G be a directed, acyclic graph with a weight function w on its 
edges, and let A = (Aj,Ao,...,An) and E = (...,E,E2,...) be sequences of vertices 
in G. 


(a) Ifnis even, then 


n/2 
ye YY GF(Le¢(A’ => E’ | non-intersecting); w) 
s=0 AICAJE/CE 

|A’|=|E’|=2s 


= Phi<icjen ((—1)*47'+1Qc(i, jsw)), (10.114) 
where Qg(i, j;w) has the same meaning as in Theorem 10.13.5. 
(b) Ifn is odd, then 


n 
ye Y GF(L¢(A’ > E' | non-intersecting); w) 
s=0 A'CA,E/CE 
|A"|=|E'|=s 


= Pe ((-1)*4 "+8 Qeli, sw), (10.115) 


ic j<n+l 


where for j <n the quantity Qc(i, j;w) has the same meaning as in Theo- 
rem 10.13.5, while Qg(i,n + 1;w) equals the generating function t~'|¥ pw(P) 
for all paths P from A; to some vertex in E. 


(c) Ifnis even, then 


t’ \  GF(L¢(A’ > E’ | non-intersecting); w) 
2 _7)Hi-1 4.72 i, ji 
oe 1) +t°Qc(i,jsw)), (10.116) 


where for j <n+1 the quantity QG(i, j;w) has the same meaning as in (b), 
and Qg(i,n+2;w) =0. 


For applications of this theorem in plane partition enumeration see [100] and 
[114]. 
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If one weakens the condition of non-intersection of lattice paths in the plane to 
the requirement that paths are allowed to touch each other in isolated points but not 
to change sides, then one arrives at the model of osculating paths. The motivation to 
consider this model comes from an observation of Bousquet-Mélou and Habsieger 
[17] that alternating sign matrices are in bijection with families of osculating paths 
with appropriate starting and end points. Alternating sign matrices are fascinating, 
but notoriously difficult to count, therefore it may be useful to investigate objects that 
are equivalent to them. So far, this point of view has not led to much, but recently 
Brak and Galleas [21] proved a constant term formula for families of osculating 
paths. 


10.14 Lattice paths and their turns 


In this section we consider turns of lattice paths. Literally, a turn of a lattice path 
is any vertex of a path where the direction of the path changes. The enumeration of 
lattice paths with a given number of turns is motivated by problems of correlated 
random walks, distribution of runs (cf. [95]), coefficients of Hilbert polynomials of 
determinantal and Pfaffian rings (cf. [84, 86]), and summations for Schur functions 
(cf. [76]). 

The approach for the enumeration of simple plane lattice paths with respect to 
their number of turns that we present here is by encoding lattice paths in terms of 
two-rowed arrays, a point of view put forward in [78]. 

For simple lattice paths in the plane there are two types of turns. We call a vertex 
T of a path a North-East turn (NE-turn for short) if T is reached by a step towards 
north and left by a step towards east. We call a vertex T of a path an East-North turn 
(EN-turn for short) if T is reached by a step towards east and left by a step towards 
north. Thus, the NE-turns of the path Pp in Figure 10.26 are (1,1), (2,3), and (5,4), 
the EN-turns of Po are (2,1), (5,3), and (6,4). We denote by NE(P) the number of 
NE-turns of P and by EN(P) the number of EN-turns of P. 

Now we describe the encoding of paths in terms of two-rowed arrays. Actually, 
we use two encodings, one corresponding to NE-turns, one corresponding to EN- 
turns. Let (p1,q1), (P2,42), ---» (pe, qe) be the NE-turns of a path P. Then the NE- 
turn representation of P is defined by the two-rowed array 


Cy We. re ee (10.117) 
qi gz +++ Qe; 
which consists of two strictly increasing sequences. Clearly, if P runs from (a,b) to 
(c,d) thena<p; <po<-:-<pe<c—landb+l1<qi<qa<-:-<qe<dare 
satisfied. If we wish to make this fact transparent, we write 


aS pr pr «+ pe <ce-l 


b+1l< 4 92 +. Qe <d. vou. 


For a given starting point and a given final point, by definition the empty array is 
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Po 


Figure 10.26 
The path Py has NE turns (1,1), (2,3), and (5,4), and it has EN turns (2,1), (5,3), 
and (6,4). 


the representation for the only path that has no NE-turn. For the path in our running 
example we obtain the NE-turn representation 


L253 
1 3 4’ 
or, with bounds included, 
1< 1 2 =°5 <5 
0< 1 3 4 <6 
Similarly, if (p1,4¢1), (p2,q2), ---, (pe, qe) denote the EN-turns of a path P, then 


(10.117) is called the EN-turn representation of P. If P runs from (a,b) to (c,d) 
thena+1< pi <pr2<---<pe<candb<q <q2.<::-<qe<d— are satisfied. 
Again, as earlier, we write 


at+l< pi pr «.» pe <e 


bs 42 +s 4 <f=1, (10.119) 


For a given starting point and a given final point, by definition the empty array is 
the representation for the only path that has no EN-turn. For the path in our running 
example we obtain the EN-turn representation 


2 5 6 
1 3 4?’ 


or, with bounds included, 
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Also two-rowed atrays with rows being of unequal length will be considered. 
These arrays will also have the property that the rows are strictly increasing. So, 
by convention, whenever we speak of two-rowed arrays, we mean two-rowed arrays 
with strictly increasing rows. For these arrays we will use a notation of the kind 
(10.118) or (10.119) as well. We shall frequently use the short notation (a | b) for 
two-rowed arrays, where a denotes the sequence (a;) of elements of the first row, and 
b denotes the sequence (b;) of elements of the second row. 

From (10.118) we see at once that the number of all paths from (a,b) to (c,d) 
with exactly @ NE-turns is in fact equal to the number of ¢-element subsets of the set 
{a,a+1,...,c—1} times the number of ¢-element subsets of {b+ 1,b+2,...,d}.A 
similar argument holds for EN-turns. Thus we have proved 


|L((a,b) — (c,d) | NE(.) = £)| = |L((a,b) > (c,d) | EN(.) = 8)| 


c—a\ (d—b 
= 2 ( 2 ), (10.120) 


A lattice path statistic that is frequently used is the number of runs of a lattice 
path. A run in a path P is a maximal subpath of P consisting of steps of equal type. 
We write run(P) for the number of runs of P. The runs of the path Pp in Figure 10.26 
are the subpaths from (1,—1) to (1,1), from (1,1) to (2,1), from (2,1) to (2,3), 
from (2,3) to (5,3), from (5,3) to (5,4), from (5,4) to (6,4), and from (6,4) to 
(6,6). Thus we have run(Fy) = 7. Obviously, the number of runs of a path is exactly 
one more than the total number of turns (both NE-turns and EN-turns). Besides, there 
is also a close relation between NE-turns and runs, which allows us to translate any 
enumeration result for NE-turns into one for runs. 

To avoid case-by-case formulations, depending on whether the number of runs is 
even or odd, we prefer to consider generating functions. Suppose we know the num- 
ber of all paths from A to E satisfying some property R and containing a given num- 
ber of NE-turns. Then we know the generating function GF (L(A — E | R);xN#), 
For brevity, let us denote it by F(A > E | R;x). We define four refinements of 
F(A—> E|R;x). Let Fy,(A > E | R;x) be the generating function Y px“) where the 
sum is over all paths in L(A — E | R) that start with a horizontal step and end with a 
vertical step. The notations Fj,,(A > E | R;x), Fyn(A > E | R;x), and Fy, (A > E | R3x) 
are defined analogously. The relation between enumeration by runs and enumeration 
by NE-turns is given by 


GF (L(A > E | R);x°™) = xFy,(A 9 E | Rix") +2°Fyy(A  E | Rix’) 
+Fy,(A > E | R3x")+xF,(A > E| 3x’). (10.121) 


All the four refinements of the NE-turn generating function can be expressed 
in terms of NE-turn generating functions. This is seen by setting up a few linear 
equations and solving them. Evidently, the following is true: 


F(A > E|R3x) = Fyy(A > E | R3x) + Fy (A E | Rx) 
+Fyy(A > E | R3x)+ Fy (A E | R3x). 
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Besides, if E; = (1,0) and Ey = (0,1) denote the standard unit vectors, we have 


Fin (A SE | R;x) + Fry (A SE | R;x) =F(A+E, oE | R;x) 
Fiy(A > E| Rx) + F(A > E| Rx) = F(A > E- Ep | R;x) 
Fiy(A > E| Rx) = F(A+E) > E- Ep | R3x). 


Solving for Fry, Fry, Fun, Fv we get 


F(A 3 E | Rjx) = F(A+E, + E| Rix) — F(A+E, + E—E|R3x) (10.122a) 
F(A > E | Rix) = F(A+E, > E—E) | R;x) (10.122b) 
F(A > E| R;x) = F(A E | Rx) +(A+E) 3 E— Ep | Rx) (10.122c) 

—F(A+E, >E|R:x)— F(A E—Ep|R;x) (10.122d) 
F(A > E | Rx) = F(A E— E> | R;x) — F(A+E, > E—E) | R;x). (10.122e) 


As we know from Section 10.3, counting paths restricted by x = y, or even by 
two lines x = y+¢ andx = y+, is effectively solved by the reflection principle. Of 
course, reflection by itself is useless for counting paths by turns, since the reflection 
of portions of paths does not take care of turns. It might introduce new turns or make 
turns disappear. However, there are “analogues” of reflection for two-rowed arrays, 
which are due to Krattenthaler and Mohanty [81]. 


Theorem 10.14.1 Let a> b and c > d. The number of all paths from (a,b) to (c,d) 
staying weakly below x = y with exactly € NE-turns is given by 


|L((a,b) + (c,d) |x>y,NE(.) = £)| 


c—a\ (d—b c—b—-1)\ /d—-a+l 
a - 10.123 
CEE) Caen ) Cra) one 
and with exactly € EN-turns is given by 
|L((a,b) + (c,d) |x>y, EN(.) = )| 
c—a\ (d—b c—b+1)\ (d—a-1 
= a . (10.124 
CEE) - CP ET |) cons 
Proof. We start with proving (10.123). By the NE-turn representation (10.118), the 


paths from (a,b) to (c,d) staying weakly below x = y with exactly ¢ NE-turns can be 
represented by 


al pr pr «=. pe <ce-l 
10.125 
b+1< M1 G2 + Ge <d, ( 2 
where 
Pi > Wis i=1,2,...,é (10.125b) 


Following the argument in the proof of Theorem 10.3.1, the number of these two- 
rowed arrays is the number of all two-rowed arrays of the type (10.125a) minus those 
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two-rowed arrays of the type (10.125a) that violate (10.125b), i.e. where p; < q; for 
some i between | and ¢. We know the first number from (10.120). 

Concerning the second number, we claim that two-rowed arrays of the type 
(10.125a) that violate (10.125b) are in one-to-one correspondence with two-rowed 
arrays of the type 


Debi po... pe <c-1 
a Tae se 10.126 
ax -@ HU H + UH <a. ( ) 
The number of all these two-rowed arrays is (oe) (Sees as desired. So it only 


remains to construct the one-to-one correspondence. 
Take a two-rowed array (p | q) of the type (10.125a) such that p; < q; for some i. 
Let J be the largest integer such that p; < q;. Then map (p | q) to 


re Gi DEEN, see. De. (10.127) 


Note that both rows are strictly increasing because of qj-1 < gq; < gi4i < PI+i 
and p; < q;. By some case-by-case analysis it can be seen that (10.127) is of type 
(10.126). For example, if J = @ then we must check g;_; < c— 1, among others. 
Clearly, this follows from the inequalities g7_) <q; <d<c. 

The inverse of this map is defined in the same way. Let (p | q) be a two-rowed 
array of the type (10.126). Let J be the largest integer such that jj; < qj. If there are 
none, take J = 2. Then map (p | q) to 


Go cree Gi-1 Prz1 +++ Be (10.128) 
T 4 


It is not difficult to check that the mappings (10.127) and (10.128) are inverses of 
each other. This completes the proof of (10.123). 
The second identity, (10.124), can be established similarly. a 


Remark 10.14.2 The above proof leads in fact to g-analogues; see [81]. 


A refinement of Theorem 10.3.3 taking into account turns may as well be derived 
in this way. 


Theorem 10.14.3 Leta+t>b>a+sandc+t>d->c+s. The number of all 
paths from (a,b) to (c,d) staying weakly below the line y = x +t and above the line 
y=x+5 with exactly € NE-turns is given by 


|L((a,b) > (c,d Le er 0)| 
ee en 


c—b—k(t—s)+s—1\ (d—a+k(t—s)—st+1 
-{ i )( i (10.129) 
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Some of the results in Section 10.4 allow also for refinements taking into account 
turns. 


Theorem 10.14.4 Let u be a positive integer and c > ud. The number of all lattice 
paths from the origin to (c,d) that stay weakly below x = Uy with exactly @ NE-turns 
is given by 


|L((0,0) + (c,d) |x > wy, NE(.) =9| = & (‘) -u(¢ ') Ce 
(10.130) 


and with exactly € EN-turns is given by 


|L((0,0) + (c,d) |x > py, EN(.) = é)| 


_e-Hdt+1fetl)\ (d—-1\ _ fetl\(d—-l)\ _ c d 
pare FT ( l \Gaa)={ l Ge) we te) one) 


Two-rowed arrays may also be used to prove this result, see [78]. A very elegant 
alternative proof using a rotation operation on paths is given by Goulden and Serrano 
[55]. 

We conclude this section by stating results on the enumeration of families of non- 
intersecting lattice paths with respect to turns. This type of problem originally arose 
from the study of the Hilbert polynomial of certain determinantal and Pfaffian rings 
(cf. [84, 86]). The results are due to Krattenthaler [75]. We do not provide proofs. 
Suffice it to mention that they work by using two-rowed arrays. 

Let A = (Aj,A2,...,An) and E = (£),Fo,...,E,) be points in Z?. How many 
families P = (P;,P),...,P,) of non-intersecting lattice paths, where P; runs from A; 
to E;, i= 1,2,...,n, are there such that the total number of NE-turns in P is some 
fixed number, f say? 

We give the following three theorems about the counting of non-intersecting lat- 
tice paths with a given number of turns. The first theorem concerns counting families 
of non-intersecting lattice paths with given starting and end points with a given num- 
ber of NE-turns. 


Theorem 10.14.5 Let A; = (a\” as) and E; = (ef, e) be lattice points satisfying 


al) < a”) Sees al”), al) > a? De: al”, 
and 
el) < e?) ert el”), ef) = e) SS ef”), 


The number of all families P = (P,,P»,...,P,) of non-intersecting lattice paths P, : 
A; — E;, such that the paths of P altogether contain exactly € NE-turns, is 


Grn, ese oe PAO Oe de 3 
y a (G ay +i ’) & és 2) abo) 
by poly =e Sh ISM &t+i-j i 


The second theorem concerns counting families of non-intersecting lattice paths stay- 
ing weakly below x = y, with given starting and end points, by their number of NE- 
turns. 
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Theorem 10.14.6 Let Aj = (a\ as) and E; = (e!,e) be lattice points satisfying 


al) < a?) Sees a”), al) > al?) PT al”, 


1 
el) < e?) << el”), ef) = e) SS ef”), 
(i) 


and al’ > al’, el!) 2 ey, 1=1,2,...,n. The number of all families P = 
(P|, P2,...,P,) of non-intersecting lattice paths P; : A; > Ej, that stay weakly be- 
low the line x = y, and where the paths of P altogether contain exactly € NE-turns, 
is 


y ie (Geee ee) 
Oph we SISA ;+i-j t; 


GO _-_ G)_ Osa: 
_ (er -% —i-j+l) (es’-a’t+i+j-1 
( tj )( oi . (10.133) 


In the third theorem (basically) the same families of non-intersecting lattice paths as 
before are counted, but with respect to their number of EN-turns. By a rotation by 
180° this could be translated into a result about counting families of non-intersecting 
lattice paths staying above x = y, with given starting and end points, with respect to 
NE-turns. 


Theorem 10.14.7 Let Aj = (a\” as) and E; = (ee) be lattice points satisfying 


AM caQcnrcal, a aa) >.> aL" 


2 


el) < 2?) Sees ol”), ef) > e) aes ef”), 


and al’ = al, el!) = ef), i= 1,2,...,n. The number of all families P = 
(P|, Po,...,P,) of non-intersecting lattice paths P,: A; —> E;, which stay weakly below 
the line x = y, and where the paths of P altogether contain exactly € EN-turns, is 


Yaet Cc. (el -a-t43) 
byte tly we ISSA fehi fj 


_ (ed —a) -i- 43) (ef —a) +14 j-3 (10.134) 
é;—j+1 &j+i-| “= 


10.15 Multidimensional lattice paths 


This section and the following three contain enumeration results for lattice paths in 
spaces of higher dimension. Most of the time, we shall be concerned with the d- 
dimensional lattice Z“. The coordinates in d-dimensional space will be denoted by 
X1,X2,...,Xq- 
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Obviously, as a basis to start with, we need the number of all simple paths in Z4 
(that is, paths consisting of positive unit steps in the direction of some coordinate 


axis) from (a1,d2,...,aq) to (€1,€2,...,@q). Since these lattice paths can be seen as 
permutations of e; — a, steps in x;-direction, e2 — a2 steps in x2-direction, ..., eg —ag 
steps in xg-direction, the answer is a multinomial coefficient, 
d 
Yi-1 (ei — ai) 
(arya) + (€1s--vea))| = ( ‘ . (10.135) 
€| —41,€2 —42,..-,€d — Ad 


10.16 Multidimensional lattice paths bounded by 
a hyperplane 


Here, we consider simple lattice paths in Z¢*! restricted by a hyperplane of the form 
Xo = ye, Ux; where pj, i=0,1,...,d, are nonnegative integers. It should be noted 
that the reflection principle does not apply because, in general, the set of steps is not 
invariant under reflection with respect to such a hyperplane (except of course when 
H; = 1 for all i, in which case the reflection principle does apply). 


Theorem 10.16.1 Let Uo, U1,...,lg be nonnegative integers and cy,C1,...,Cq inte- 
Ho, HM1,---,H & & CL yeees 

gers such that co = yd. Lich: The number of all lattice paths from the origin to 

(co,C1,---,€a) not crossing the hyperplane x9 = yt Hix; is given by 


_ co— LE mci +1 ( 14+ Dhoci ) 


142% 0¢; cot 1,c1,€2,---5Ca 
(10.136) 


d 
L(0 > (co,€1,---,Ca) | x0 > y? uixi) 
i=1 


i= 


We omit the proof. Both proofs of Theorem 10.4.5, the generating function proof 
and the proof by use of the cycle lemma, can be extended to proofs of the above 
theorem. 

To conclude this section, we point out that Sato [105] has extended his generating 
function results for the number of paths in the plane integer lattice between two par- 
allel lines that we presented in Section 10.5 to the multidimensional case. Similarly, 
the result of Niederhausen on the enumeration of paths in the plane integer lattice 
subject to a piece-wise linear boundary, which was presented in Section 10.6, has a 
multidimensional extension; see [98, Sec. 2.2]. 


10.17 Multidimensional paths with a general boundary 


In this section we generalize the enumeration problem of Section 10.7 to arbitrary di- 
mensions. Let nj,n2,...,ng be nonnegative integers and a and b be increasing integer 
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functions defined on the box 
d 
(0, n] = [[{o. 1,.. : ni} 
i=l 


such that a > b. a is increasing means that a(i) < a(j) whenever i < j in the usual 
product order. We ask for the number of all paths in Z“+! from (0,b(0)) to (n,a(n)) 
that always stay in the region “that is bounded by a and b,” by which we mean the 
region 

{(i,y) : BG) <y< ali}. (10.137) 


The generalization of Theorem 10.7.1, due to Handa and Mohanty [62], reads as 
follows. 


Theorem 10.17.1 Let nj,12,...,ng be nonnegative integers and p = Thy ni. As- 
sume that the points in the box [0,n| are 0 = uo, U1,U2,...,Up =n, ordered lexico- 
graphically. Then the number of all lattice paths in Z¢*' from (0,b(0)) to (n,a(n)) 
that always stay in the region (10.137) equals 


|L((0,b(0)) + (n,a(n)) |a > y > b)| 
= (12h det Ces (10.138) 


0<i,j<r4 , nj-1 Uj41 —Ui 


The most elegant and illuminating proof is by the use of non-intersecting lattice 
paths, see [115]. Sulanke proves in fact a g-analogue in [115]. 


10.18 The reflection principle in full generality 


We have explained the reflection principle in the proof of Theorem 10.3.1 in Sec- 
tion 10.3, where it solved the problem of counting simple lattice paths in the plane 
bounded by the diagonal. Nothing prevents us from applying the same idea in a 
higher-dimensional setting. It is then natural to ask: How far can we go with the re- 
flection principle? What is the most general situation where it applies? This question 
was raised and answered by Gessel and Zeilberger [53], and, independently, by Biane 
[10] in a more restricted setting; see also Grabiner and Magyar [57]. 

The standard example, which will serve as our running example, is the problem 
of counting all paths from (a1,a2,...,aq) to (e1,¢2,...,@q) that always stay in the 
region x; > x2 > +--+: > xg. This problem is equivalent to several other enumeration 
problems, the most prominent being the d-candidate ballot problem (for the 2- 
candidate ballot problem see Section 10.3) and the problem of counting standard 
Young tableaux of a given shape. 

In the d-candidate ballot problem there are d candidates in an election, say 
E\,Fo,...,Eq, E\ receiving e; votes, E> receiving e2 votes, ..., Eg receiving eg votes. 
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How many ways of counting the votes are there, such that at any stage during the 
counting EF has at least as many votes as E>, E> has at least as many votes as £3, 
etc.? It is evident that by encoding each vote for candidate EF; by a step in x;-direction 
this ballot problem is transferred into counting paths from the origin to (e1,¢2,...,€a) 
that are staying in the region x] > x2 >--- > xq. 

A standard Young tableaux of skew shape A /u, where A = (A,,A2,..., Aq) and 
= (U1, 2,---, Ha) are d-tuples of non-negative integers that are in non-increasing 
order and satisfy A; > 4; for all i, is an arrangement of the numbers 1,2,... ip Dae (A;— 
i) of the form 


Md gt] tre t eee eee eee NdAg 


such that numbers along rows and columns are increasing. See Chapter 14 for more 
information on these important combinatorial objects. By encoding an entry i located 
in row j of the tableau by a step in x;-direction, i= 1,2,...,Y., (A; — Mj), it is easy 
to see that standard tableaux of shape A/p are in bijection with lattice paths from 
MW = (U1, bb,---,;Ma) to A = (Ay, A2,...,Ag) consisting of positive unit steps in the 
direction of some coordinate axis, and that stay in the region x} > x2 >--- > xq. 

It is a classical result due to MacMahon [90, p. 175] (see also [91, §103]) that the 
solution to the counting problem is given by a determinant, see e.g. [112, Prop. 7.10.3 
combined with Cor. 7.16.3]. 


Theorem 10.18.1 Let A = (aj,a2,...,ag) and E = (e},e2,...,eq) be points in Z4 
with a, > a2 > +++ > ag and e, > e2 > --: > eg. The number of all lattice paths from 
A to E consisting of positive unit steps in the direction of some coordinate axis and 
staying in the region x1 > x2 > ++: > Xq equals 


LAE | 2m>—2ayl=(Y(e—a))! det (<==): 


= 1<i,j<d \ (e; aj —i+j)! 
(10.139) 


If the starting point A equals the origin, then the above determinant can be re- 
duced to a Vandermonde determinant by elementary column operations, and thus it 
can be evaluated in closed form. If one rewrites the result appropriately, then one 
arrives at the celebrated hook formula due to Frame, Robinson and Thrall [45]. (We 
refer the reader to [103, Sec. 3.10] or [112, Cor. 7.21.6] for unexplained terminol- 


ogy). 


Theorem 10.18.2 Let E = (e1,e2,...,eq) be a point in Z4 with e; > e2 > ++» >eqg> 
0. The number of all lattice paths from the origin to E consisting of positive unit steps 
in the direction of some coordinate axis and staying in the region x1 > x2 > +--+ > Xq 
equals 


(cde)! 
ee ee Se) = age) (10.140) 
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where the product is over all cells p in the Ferrers diagram of the partition 
(€1,€2,-..,€4), and h(p) is the hook-length of the cell p. 


It was pointed out by Zeilberger [124] that the formula in (10.139) can be proved 
by means of the reflection principle. The natural environment for a “general reflec- 
tion principle” is within the setting of reflection groups. A reflection group is a 
group that is generated by all reflections with respect to the hyperplanes H in a given 
set # of hyperplanes (in some R@). We review the facts about reflection groups that 
are relevant for us below. For an excellent exposition of the subject see Humphreys 
[63]. As we already said, the situation of Theorem 10.18.1 will be our running 
example. 

As above, let # be a (finite) set of hyperplanes in some R?. Let W denote the 
group that is generated by the corresponding reflections. By definition, W is a sub- 
group of O(d). Some of the elements of W happen to be reflections with respect to a 
hyperplane (not necessarily belonging to #), and let Z denote the collection of all 
these hyperplanes. Of course, Z contains #. In the example when .# is the set of 
hyperplanes H; given by 


Hi: xj—%41=0, i=1,2,...,d—1, (10.141) 


(these are the hyperplanes restricting the paths in Theorem 10.18.1), the group W is 
the permutation group Gy, acting on R?@ by permuting coordinates. All the reflec- 
tions in this group are the interchanges of two coordinates x; and xj, 1 <i<j<d, 
corresponding to the transpositions (i, /) in Gg. Hence, the corresponding set & of 
hyperplanes in this case is 


RB ={xj—xj =0: 1<i< j<d}. (10.142) 


The hyperplanes in # cut the space into different regions. The connected com- 
ponents of the complement of U,<.gH in R¢ are called chambers. Each chamber is 
enclosed by a set Zp of bordering hyperplanes. Clearly, Zp is a subset of Z. In our 
running example a typical chamber is the region 


{(x1,%2,---,Xa): xX] > x2 > +++ > xg}, (10.143) 


bounded by the hyperplanes in (10.141). As a matter of fact, in this special case any 
chamber has the form 


{(x1,%2,... Xd) » Xo(1) > Xo(2) Po > Xo(a)}s (10.144) 


where o is some permutation in Gy. 

It can be shown that the reflections with respect to the hyperplanes in Zp generate 
the complete reflection group W. Another important fact is that, given one chamber 
C, all chambers are w(C), where w runs through the elements of the reflection group 
W, all w(C)’s being distinct. 

Now we are in the position to formulate and prove Gessel and Zeilberger’s result 
[53, Theorem 1]. The motivation for the technical conditions in the statement of the 
theorem involving ky and ry is that they make sure that it is not possible to “jump” 
over a hyperplane without touching it in a lattice point. 
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Theorem 10.18.3 Let C be a chamber of some reflection group W, determined by 
the hyperplanes in the set Zo. Let S be a set of steps that is invariant under W, 
i.e, w(S) =S, and with the property that for all hyperplanes H € &o and all steps 
s €§ the Euclidean inner product (s,ry) is either 0 or tky, where ky is a fixed 
constant, ry is a fixed non-zero vector perpendicular to H, both depending only on 
the hyperplane H. Furthermore, let A and E be lattice points inside the chamber C 
such that also w(A) and w(E) are lattice points for all w € W, and such that for all 
hyperplanes H € &o the Euclidean inner product (A,ry) is an integral multiple of 
ky. 

Then the number of all lattice paths from A to E, with exactly m steps from S, and 
staying strictly inside the chamber C, equals 


\Lin(A + E;S | inside C)| = y (sgnw) |Lin(w(A) > E;S)|, (10.145) 


wew 
where sgnw = detw, considering w as an orthogonal transformation of R4. 


Remark 10.18.4 A weighted version of the above theorem in which steps carry 
weights such that images of steps under W carry the same weight holds as well. 


Proof. We may rewrite (10.145) in the form 


[Lmn(A + E;S | inside C)| = )° sgnw, (10.146) 
(w,P) 


where the sum is over all pairs (w,P) with w € W and P € L,,(w(A) > E;S). The 
proof of (10.146) is by a sign-reversing involution on the set of all such pairs (w, P), 
where P touches at least one of the hyperplanes in #%. Sign-reversing has to be 
understood with respect to sgnw. Provided the existence of such an involution, the 
only contributions to the sum in (10.146) would be by pairs (w,P) where P does 
not touch any of the hyperplanes in #. We claim that this can only be the case for 
w = id. In fact, as we already mentioned, it is one of the properties of a reflection 
group W that, given one chamber C, all chambers are w(C), w € W, and all w(C)’s 
are distinct. Therefore, if A is in C and w # id, then w(A) must be in a different 
chamber and so cannot be in C. Thus, evidently, any path from w(A) to E, the point 
E being inside C, must touch at least one of the bordering hyperplanes. This would 
prove (10.146) and hence the theorem. 

Now we construct the promised involution. Fix some order of the hyperplanes in 
HQ. Let (w,P) be a pair with w € W, P EL, (w(A) > E;S), and P touching at least 
one of the hyperplanes in .#. Consider all meeting points of P with hyperplanes in 
2. Choose the last meeting point along the path P and denote it by M. M must be a 
lattice point because of the assumptions that involve the constants kz. Let H be the 
first hyperplane (in the chosen order) that meets P. Then we form the new path P’ by 
reflecting the portion of P from the starting point w(A) up to M with respect to H and 
leaving the portion from M to P invariant. By assumption, reflection of a step from S 
is again a step in S. So, P’ also consists of steps from S only. Evidently, the starting 
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point of P’ is wyw(A), where w;; denotes the reflection with respect to H. Hence, 
(waw, P’) is a pair under consideration for the sum in (10.146), and P’ touches one 
of the hyperplanes in “ (namely H). This mapping is an involution since nothing 
was changed after M. Moreover, we have sgnwyw = —sgnw. Therefore it is also 
sign-reversing. This completes the proof of the theorem. | 


In the case of our running example, W is the group of permutations of coordi- 
nates, C is given by (10.143), the set of hyperplanes is (10.141), the set of steps is 


{€|,&,...,€ }, with €; denoting the positive unit vector in x;-direction. If H; is the 
hyperplane x; — x;;; = 0, we may choose ry, = €; — €+1, so that all constants ky, 
are 1,i=1,2,...,d —1. Since the number of lattice paths between two given lattice 


points is given by a multinomial coefficient (see (10.135)), it is then not difficult to 
see that (10.145) yields (10.139) in this special case. 

Which other examples are covered by the general setup in Theorem 10.18.3? The 
answer is that all reflection groups that are “relevant” in our context are completely 
classified. The meaning of “relevant” is as follows. In order for formula (10.145) to 
make sense, the sum on the right-hand side of (10.145) should be finite. So, only 
reflection groups that are “discrete” and act “locally finite” will be of interest to us. 
It is exactly these reflection groups that are precisely known (see Humphreys [63, 
Sec. 4.10], Bourbaki [15, Ch. V, VI]). 

The classification of all finite reflection groups says that any such finite reflec- 
tion group decomposes into the direct product of irreducible reflection groups, all of 
which act on pairwise orthogonal subspaces. These irreducible reflection groups do 
not decompose further. There exist four infinite families of types 4(m) (m= 1,2,...), 
Aq, Ba = Cg, Dg (d = 1,2,...) of such groups, and the seven exceptional groups of 
types Go, Fa, Ee, E7, Eg, and H3, H4. (The indices mark the dimension of the vector 
space on which they act faithfully.) In addition, for most of these irreducible finite 
reflection groups there exists an affine reflection group, which is infinite. The finite 
reflection group is generated by all reflections with respect to the hyperplanes that 
run through a given point (we assume that this is the origin). The affine reflection 
group is generated by a larger set of hyperplanes, which includes the aforementioned 
hyperplanes plus certain translates of them. The reflection groups corresponding to 
G» are the same as those for I)(6), therefore we need not consider G). 

Grabiner and Magyar [57, p. 247] have determined all possible step sets (up to 
dilation) for each of the irreducible reflection groups such that the technical con- 
ditions of Theorem 10.18.3 are satisfied. Not for all types do there exist such step 
sets. It should be noted however that the “empty” step (0,0,...,0) can always be 
added to any possible step set. The following list describes all possible instances 
of Theorem 10.18.3 when applied to an irreducible finite or affine reflection group. 
The results for lattice paths in chambers of affine reflection groups have been made 
explicit by Grabiner [56]. 


e Types H3, H4, Fy, Eg, (m): There are no possible step sets. 


e Type Ay_;: The set of reflecting hyperplanes is Z = {xj-xj =0: 1<i<j< 
d}. Obviously, the reflection with respect to x; — xj; = 0 acts by interchanging 
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the ith and jth coordinate. So, the associated finite reflection group is the 
group of permutations of the coordinates x1,x2,...,xg, which is isomorphic 
to the permutation group Gy. A typical chamber is C = {(x1,x2,...,Xq) : 
X] > x2 > +++ > xg}. 


Possible step sets are the sets 
Sp:={w-(1,...,1,0,...,0):we Ga}, k=1,2,...,d 


(with & occurrences of 1), all compatible with each other, as well as 


S* := {w-(+1,...,+1,0,...,0): we Gy}, k=landk=d 


(with & occurrences of +1), which can not be mixed together. 


Theorem 10.18.1 is a direct consequence of Theorem 10.18.3 with W = Gg and 
S=S. 
The second standard application is the one for S = S7. 


Theorem 10.18.5 Let A = (a,@,...,aq) and E = (e1,€2,...,eq) be points in Z4, 
with all a;’s of the same parity, all e;’s of the same parity, a, > az > ++: > ag and 
e) > e2 >--: > eg. The number of all lattice paths from A to E consisting of m steps 
from SZ and staying in the region x; > x2 > +++ > Xq equals 


|L(A > E;SF | x1 > 22 >+++ >xa)| =. det ( (nse) ) (10.147) 
1<i,j<d — 9 
We should point out that the lattice paths in Theorem 10.18.5 are in bijection with 
configurations in the lock step model, a frequently studied vicious walker model. 
On the other hand, the lattice paths in Theorem 10.18.1 are in bijection with configu- 
rations in another popular vicious walker model, the so-called random turns model. 
We refer the reader to [80, Sec. 2] for more detailed comments on these connections. 
The associated affine reflection group, the affine reflection group of type Ay_1, is 
generated by the reflections with respect to the hyperplanes Z = {xj -—xj =k: 1< 
i< j<d, k © Z}. The elements of this group are called affine permutations. They 
act by permuting the coordinates x;,x2,...,%q and adding a vector (ki,k2,...,kg) 
with kj +k2 +---+kg =0. A typical chamber * is C = {(x1,%2,...,%a) 2 X1 > x2 > 
+++ >Xq > x, — 1}. For enumeration purposes, we inflate this chamber, see (10.148) 
below. 
The probably first explicitly stated enumeration result for lattice paths in an affine 
chamber is the result below due to Filaseta [39], although it was not formulated in 
that way. 


Theorem 10.18.6 Let A = (a\,a2,...,aq) and E = (e},e2,...,e4) be points in Z4 
with a, > a2 >-+: >agande, > e2 >-++: > eg. The number of all paths from A to E 
consisting of steps from S, and staying in the chamber 


{(x1,%2,---,Xg) 1X1 > XQ > ++ > Ag > x1 —N} (10.148) 


*Actually, the chambers of affine reflection groups are usually called alcoves. 
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of type Ag_1, equals 


|IL(A > E3Sy | x1 > x2 > +++ > xg > x1 —N)| 


=(Yte-a))! y det (cm): (10.149) 


i=l ky +k =0 1SbIS4 


See [82] for a g-analogue. It should be noted that Theorem 10.3.3 follows from 
the special case of the above theorem where d = 2. 

For the step set S| consisting of positive and negative unit steps in the direction 
of some coordinate axis, we obtain the following result. 


Theorem 10.18.7 Letm and N be positive integers. Furthermore, let A= (a),a2,..., 
aq) and E = (e1,€2,...,@a) be vectors of integers in the chamber (10.148) of type 
Aq_1. Then the number of lattice paths from A to E with exactly m steps from Ss); 
which stay in the alcove (10.148), is given by the coefficient of x" /m!\ in 


det Tsu (2x)) 10.1 
byt egw StS ej a;+Nk; ( x) ry (10 50) 


where Iq(x) is the modified Bessel function of the first kind 


o0 G/2yare rite 


— Lngrayl !(j+a)! 


The last result for type Ag_; which we state is the one for paths in an affine 


chamber of type Ag_; with steps from Sy: 


Theorem 10.18.8 [56, Eq. (35)] Let m and N be positive integers. Furthermore, let 
A = (a1,a2,...,dq) and E = (e1,€2,...,e@qa) be vectors of integers in the chamber 
(10.148) of type Ag_ such that all a;’s have the same parity, and all e;’s have the 
same parity. Then the number of lattice paths from A to E with exactly m steps from 
S7 which stay in the chamber (10.148), is given by 


|Lm(A + E;S7 | xy > XQ > > XY >x1—N)| 


m 
= det . (10.151 
sa ipa Ks (Gz “+ Nk; )) : 


e Types By, Cy: The finite reflection groups of types By and Cg are identical. 
The set of reflecting hyperplanes is @ = {+xj4xj =0: 1<i<j<d}U 
{xi =0:1<i<d}. Obviously, the reflection with respect to x; —x; = 0 
acts by interchanging the ith and jth coordinate, the reflection with respect 
to x; +x; = 0 acts by interchanging the ith and jth coordinate and changing 
the sign of both, while the reflection with respect to x; = 0 acts by changing 
the sign of the ith coordinate. 


Here, the possible step sets are only S} and S7. 
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The associated finite reflection group is the group of signed permutations of 
the coordinates x) ,x2,...,Xg, which acts by permuting and changing signs of 
(some of) the coordinates x1 ,x2,...,xg. It is frequently called the hyperocta- 
hedral group since it is the symmetry group of a d-dimensional octahedron. 
It is furthermore isomorphic to the semidirect product (Z/2Z)¢ x Gq. A typ- 
ical chamber is C = {(x1,x2,.-.,Xq) 1X1 >X2 > +++ >Xq > Of. 


We have the following enumeration result for lattice paths staying in this cham- 
ber. 


Theorem 10.18.9 Let A = (a},@2,...,aq) and E = (e1,€2,...,eq) be points in Z4, 
with all a;’s of the same parity, all e;’s of the same parity, a, > az >--: > aq > Oand 
e€ > e2 >-++ > eg > 0. The number of all lattice paths from A to E consisting of m 


steps from S7, and staying in the region x, > x2 > +++ > xq > 0 equals 


+ m m 
|Lin(A — E;S7 | x1 >x2>-- > xa)| = et, (4) _ Ce) : 
ShIS 2 2 
(10.152) 


The associated affine reflection group now comes in two flavors, types By and Cy. 
A typical chamber of types Cg is C = {(x1,%2,...,%¢): 1 > x1 >x2 >+++ > xq > 0}, 
while a typical chamber of type By is C = {(x1,%2,...,%@) 1 X1 >X2 >++* > xX > 
0 and x1 +42 < 1}, 

Next we quote the two results from [56] on the enumeration of lattice paths in 
chambers of type Cy. 


Theorem 10.18.10 [56, Eq. (23)] Let m and N be positive integers. Furthermore, let 
A = (a1,d2,...,aa) and E = (e1,€2,...,ea) be vectors of integers in the chamber 


{(x1,%2,---,;4n):N > x1 > x2 > +++ > Xq > OF (10.153) 


of type Cg. Then the number of lattice paths from A to E with exactly m steps from 
Sy, which stay in the chamber (10.153), is given by the coefficient of x" /m! in 


1 2N-1 ; tra; 
det — y sin = -sin i “exp (2xc0s =) : (10.154) 
I<i,j<d \N = N N N 


The result for lattice paths with steps from S7, is the following. 
Theorem 10.18.11 [56, Eq. (18)] Let m and N be positive integers. Furthermore, 
let A = (a),d2,...,aq) and E = (e1,€2,...,€q) be vectors of integers in the chamber 
(10.153) of type Cg such that all a;’s are of the same parity, and all e;’s are of the 
same parity. Then the number of lattice paths from A to E with exactly m steps from 
S7 which stay in the chamber (10.153), is given by 


DS PAE Sa EN. 2 ape 
eee (= nan W - sin WT “cos” 5 . (10.155) 


Lattice Path Enumeration 667 


Enumeration results for lattice paths in a chamber of type B, can also be derived 
from Theorem 10.18.3. We omit to state them here, but instead refer to [56] and [80, 
Theorems 8 and 9]. 


e Type Dz: The set of reflecting hyperplanes is @ = {+xj+xj =0: 1<i< 
j <d}. Obviously, Dy is a subset of By or Cg. The action of the reflection 
with respect to a hyperplane +x; + x; = 0 was already explained there. 


The associated finite reflection group is the group of signed permutations of 
the coordinates x;,x2,...,xy with an even number of sign changes. It acts 


by permuting the coordinates x;,x2,...,xqg and changing an even number of 
signs thereof. A typical chamber is C = {(x1,%0,...,%¢@) 1 X41 > 42 >°+°: > 
Xa—1 > |Xal}- 


The associated affine reflection group is generated by the reflections with re- 
spect to the hyperplanes Z = {txjtxj =k: 1<i<j<d,k eZ}. The 
elements of this group act by permuting the coordinates x;,x2,...,%g, chang- 
ing an even number of signs thereof, and adding a vector (k1,k2,...,kq) with 
ky +ko +-+-+kg =0 (mod 2). A typical chamber is C = {(x1,x2,...,Xa) : 
Xp > XQ > +++ > Xg_-1 > |xa|, and x; +.x2 < 1}. 


We omit the explicit statement of enumeration results for types Dz and Dy, which 
one may derive from Theorem 10.18.3, and instead refer to [56] and [80, Theo- 
rems 10 and 11]. 


e Types EF, and £7: There are possible step sets (see [57, p. 247]), but since 
this does not yield interesting enumeration results, we refrain from discussing 
these two cases further. 


A non-example for the application of the reflection principle has been discussed 
in Section 10.12, see Theorem 10.12.3. 


10.19 g-Counting of lattice paths and 
Rogers—Ramanujan identities 


In this section, we discuss some q-analogues of earlier (plain) enumeration results, 
and we briefly present work showing the close link between lattice path enumeration 
and the celebrated Rogers-Ramanujan identities. 

As we have already seen in the introduction, one source of g-analogues is area 
counting of lattice paths. This idea has also been used to construct a qg-analogue of 
Catalan numbers. Given a Dyck path P (see Section 10.8) from (0,0) to (27,0), 
let G@(P) := 5 (a(P) —n). In other words, G(P) is half of the area between P and the 
“Jowest” Dyck path from (0,0) and (27,0), that is, the zig-zag path in which up-steps 
and down-steps alternate. Alternatively, @(P) counts the squares with side length V2 
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Figure 10.27 
A Dyck path. 


(rotated by 45°) that fit between P and the zig-zag path. In Figure 10.27 this zig-zag 
path is indicated as the dotted path. For the Dyck path shown with full lines in the 
figure, we have G(.) = 6. This (modified) area statistics is now used to define the q- 
Catalan number C,,(q) as the generating function for Dyck paths of length 2 with 
respect to the statistics a(.), 


Cu(q) = GF (L((0,0) — (2n,0);{(1,1), (1,-1)}):4™). (10.156) 
By decomposing a given Dyck path P uniquely into 
P=5yPiSaP2, 


where s,, denotes an up-step, sy denotes a down-step, and P; and P) are Dyck paths, 
one obtains the recurrence 


n-1 
Cn(q) = Ya Cx(Q)Cn-e-1(9), 2 > 1, (10.157) 
k=0 


with initial condition Co(g) = 1. These q-Catalan numbers have been originally in- 
troduced by Carlitz and Riordan [25]. We shall say more about these further below. 
A different statistics can be derived from turn enumeration (cf. Section 10.14). 
In the geometry we are considering here, turns are peaks and valleys of a Dyck path. 
For a peak at lattice point S, denote by x(S) the number of steps along the path 
from the origin to S. (Equivalently, x(S) is the ordinate of S.) In the Dyck path in 
Figure 10.27, the peaks are at (2,2), (5,3), (10,2), and (12,2). The ordinates are 
x((2,2)) =2,x((5,3)) =5, x((10,2)) = 10, x((12,2)) = 12, The major index of a 
Dyck path P, denoted by maj(P), is the sum of all values x(S) over all peaks S of P. 
For the Dyck path in Figure 10.27, we have maj(.) =2+5+10+12 = 29. Fiirlinger 
and Hofbauer [47] used this statistic to define alternative g-Catalan numbers, namely 


cn(q) = GF (L((0,0) + (2n, 0); {(1, 1), (1,-D)})sg™). (10.158) 
They showed that 


l1—q |2n 
ond) = at | | ; (10.159) 
q 
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the “natural” g-analogue of the Catalan number in view of its explicit formula 
oa c oe More on these g-Catalan numbers and further work in this direction can 
be found in [47, 74, 81]. These ideas have been extended to Schroder paths and num- 
bers by Bonin, Shapiro and Simion in [11], 

Returning to the g-Catalan numbers of Carlitz and Riordan, we see that by the 
choice of b; = 0,i=0,1,..., A; =q'~'z, i= 1,2,... in Theorem 10.9.1, we obtain a 
continued fraction for the generating function of g-Catalan numbers C,,(q), namely 


yC@e= (10.160) 
n=0 1 z 
z 
ie aes 
2 
1— 3 
z 
2 q 
i 
If one substitutes z = —q in this continued fraction, then it becomes the reciprocal 


of the celebrated Ramanujan continued fraction (cf. [2, Ch. 7]) 


s s. (10.161) 


where (0;q)n := (1 —@)(1—aq)---(1—aq""!), n> 1, and (3g) := 1. 
Numerator and denominator on the right-hand side of this identity feature in the 
equally celebrated Rogers—Ramanujan identities (cf. also [2, Ch. 7]) 


2 


a : (10.162) 
4 (asda (Gg?) (Gt) 
and 
oo on (n+1) 1 
Li (10.163) 


(G:Q)n  (q?sq>)x (G35) 


The fact that we came across the left-hand sides of these identities by starting with 
lattice path counting problems may indicate that the Rogers-Ramanujan identities 
themselves may be linked with lattice path enumeration. Bressoud [24] was the first 
to actually set up such a link. Since then, this connection has been explored much 
further and extended in various directions, particularly so in the physics literature, 
see [7, 29, 34, 93, 122] and the references therein. 
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10.20 Self-avoiding walks 


A path (walk) in a lattice in d-dimensional Euclidean space is called self-avoiding if 
it visits each point of the lattice at most once. One cannot expect useful formulas for 
the exact enumeration of self-avoiding paths (except in extremely simple lattices). 
This is the reason why, with a few exceptions, research in this area concentrates on 
asymptotic counting: how many self-avoiding walks are there in a particular lattice, 
consisting of n steps from a given step set, asymptotically as n tends to infinity? This 
is a notoriously difficult question, that has been investigated mainly in the physics 
and probability literature. In fact, the self-avoiding walk constitutes a fascinating, 
vast subject area, which would need a chapter by itself. We refer the reader to the 
standard book [92], and to the more recent volumes [59, 67] which contain more 
recent material on or relating to self-avoiding walks. 
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11.1 Introduction 


This chapter contains an account of a two-parameter version of the Catalan num- 
bers, and corresponding two-parameter versions of related objects such as parking 
functions and Schréder paths, which have become important in algebraic combina- 
torics and other areas of mathematics as well. Although the original motivation for 
the definition of these objects was the study of Macdonald polynomials and the rep- 
resentation theory of diagonal harmonics, in this account we focus only on the com- 
binatorics associated to their description in terms of lattice paths. Hence this chapter 
can be read by anyone with a modest background in combinatorics. In Section 11.2 
we include basic facts involving g-analogues, permutation statistics, and symmetric 
functions, which we need in later sections. Sections 11.3, 11.4, and 11.5 contain the 
results on the q,t-versions of the Catalan numbers, parking functions, and Schroder 
paths, respectively. Section 11.6 contains a brief account of the recent exciting exten- 
sions of these objects that have arisen in the study of string theory, knot invariants, 
and the Hilbert scheme from algebraic geometry. 


11.2 Introduction to g-analogues and Catalan numbers 
11.2.1 Permutation statistics and Gaussian polynomials 


In combinatorics a g-analogue of a counting function is typically a polynomial in g 
that reduces to the function in question when g = 1, and furthermore satisfies versions 
of some or all of the algebraic properties, such as recursions, of the function. We 
sometimes regard g as areal parameter satisfying 0 < g < 1. We define the g-analogue 
of the real number x, denoted [x], as 


l-q¢ 

l-q 

Work supported by NSF grant DMS-1200296. Much of this chapter is a condensed version of Chapters 
1, 3, 4, and 5 of the author’s book The q,t-Catalan Numbers and the Space of Diagonal Harmonics: With 
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By l’H6pital’s rule, [x] + x as g > 17. Let N denote the nonnegative integers. For 
n €N, we define the g-analogue of n!, denoted [n]!, as 


n 


i=] =A +a ta+e)Utat. ta), 


We let |S| denote the cardinality of a finite set S. By a statistic on a set S we 
mean a combinatorial rule that associates an element of N to each element of S. A 
permutation statistic is a statistic on the symmetric group S,. We use the one-line 
notation 0; 02 --- O, for the element o = ( a of S,. More generally, 

O,; O2 ... On 

a word (or multiset permutation) 0; 02---O, is a linear list of the elements of some 
multiset of nonnegative integers. (The reader may wish to consult [60, Chapter 1] 
for more background on multiset permutations.) An inversion of a word o is a pair 
(i,j), 1<i< j <n such that 6; > oj. A descent of o is an integer i, 1 <i<n—1, 
for which 0; > 0;+1. The set of such i is known as the descent set, denoted Des(o). 
We define the inversion statistic inv(o) as the number of inversions of o and the 
major index statistic maj(o) as the sum of the descents of o, ie., 


inv(o) = y 1, maj(o) = >: i. 
O50; Oi>Gi41 


For example, inv(613524) = 8, while Des(613524) = {1,4} and maj(613524) = 5. 
A permutation statistic is said to be Mahonian if its distribution over S,, is [n]!. 


Theorem 11.2.1 Both inv and maj are Mahonian, i.e., 


y gi) = [n]! -_ y gt), 


O€Sy OESn 


Proof. Given B € S,—1, let B(k) denote the permutation in S,, obtained by insert- 
ing n between the (k — 1)th and kth elements of 8. For example, 2143(3) = 21543. 
Clearly inv(B(k)) = inv(B) +n—k, so 


y gin() = y (poe sehen 
OESn BESp—-1 


and thus by induction inv is Mahonian. 
A modified version of this idea works for maj. Say the descents of B € S,—1 are 
at places ij <in <---<i;. Then 


maj(B(n)) = maj(B), —maj(B (is + 1)) = maj(B) +1, 
...,maj(B (i; +1)) = maj(B)+s, maj(B(1)) =s+1. 


If the non-descents less than n — 1 of B are at places a < 2 < +++ < Qy_2~s, then 


maj(B (a +1)) = maj(B) +5— (a) —1) +o, +1 = maj(B) +542. 
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To see why, note that s — (a, — 1) is the number of descents of f to the right of a, 
each of which will be shifted one place to the right by the insertion of n just after 
Ba, . Also, we have a new descent at a +1. By similar reasoning, 


maj(B (o)) = maj(B) +s — (2 —2) + a +1 = maj(B) +s+3, 


maj(B (O%,—2-s)) = maj(B) +s —(Qn-2-s— (n-2—s)) +O,-2-s +1 


= maj(B)+n-1. 
Thus 
Yg@iO= Y (tat...tg tat +...4g"'grh) 
OESy BES,-1 
and again by induction maj is Mahonian. | 


Major P. MacMahon introduced the major-index statistic and proved it is Maho- 
nian [54]. Foata [19] found a map ©® that sends a permutation o with a given major 
index to another ¢(o) with the same value for inv. Furthermore, if we denote the 
descent set of o~! by Ides(o), then Ides(@(o)) = Ides(o). The image ¢ of the map 
® can be described as follows. If n < 2, 6(0) = 0. If n > 2, we add a number to 
@ one at a time: Begin by setting o(!) = O01, o?) = 0102 and 3) = 010203. Then 
if O2 > 03, draw a bar after each element of o°) that is greater than 03, while if 
O2 < 03, draw a bar after each element of o3) that is less than 03. Also add a bar 
before go), For example, if o = 4137562 we now have o°3) = |41|3. Now regard the 
numbers between two consecutive bars as “blocks,” and in each block, move the last 
element to the beginning, and finally remove all bars. We end up with 9) = 143. 

Proceeding inductively, we begin by letting o) be the result of adding o; to the 
end of gf), Then if 0;_; > 0;, draw a bar after each element of og) that is greater 
than o;, while if o;_; < 0;, draw a bar after each element of hg) that is less than 
o;. Also draw a bar before g(. Then in each block, move the last element to the 
beginning, and finally remove all bars. If o = 4137562 the successive stages of the 
algorithm yield 


gS) = 143 

o) = |1|4|3|7 > 1437 

@°) = |1437|5 — 71435 

°) = |71|4|3|5|6 > 174356 
7) = |17|4|3|5|6|2 7143562 


and so @(4137562) = 7143562. 


Proposition 11.2.2 The equality maj(o) = inv(¢(o)) holds. Furthermore, Ides(o) = 
Ides(@(o)), and also ¢(0) and o have the same last letter. 
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Proof. We claim inv(¢“) = maj(o, --- 0) for 1 < k <n. Clearly this is true for 
k <2. Assume it is true for k < j, where 2 < j <n. If oj-1 > oj, maj(o)--- oj) = 
maj(0)---oj;-;) + j— 1. On the other hand, for each block arising in the procedure 
creating o), the last element is greater than o;, which creates a new inversion, and 
when it is moved to the beginning of the block, also creates a new inversion with 
each element in its block. It follows that inv(@“)) = inv(@Y—") + j — 1. Similar 
remarks hold if oj; < 6;. In this case maj(0j --- Oj) = maj(0) ---o;). Also, each 
element of @ that is not the last element in its block is larger than 0;, which creates 
a new inversion, but a corresponding inversion between this element and the last 
element in its block is lost when we cycle the last element to the beginning. Hence 
inv(@ 4!) = inv(@)) and the claim follows. 

Note that Ides(o) equals the set of all i, 1 <i <n such that i+ 1 occurs before 
iin o. In order for the @ map to change this set, at some stage, say when creating 
o), we must move / from the end of a block to the beginning, passing i— 1 ori+ 1 
along the way. But this could only happen if 0; is strictly between i and either i— 1 
or i+ 1, an impossibility. | 


We now show that the map ® is invertible by constructing the permutation B = 
®~!(o). Begin by setting B“) = o. Then if 6, > 6), draw a bar before each number 
in Bo) which is less than o,,, and also before 6; If 6, < 0;, draw a bar before each 
number in B () which is greater than 0, and also before o,,. Next move each number 
at the beginning of a block to the end of the block. 

The last letter of B is now fixed. Next set B'?) = B), and compare the (n — 1)th 
letter with the first, creating blocks as above, and draw an extra bar before the (n — 
1)th letter. For example, if o = 7143562 the successive stages of the B algorithm 
yield 


') — |71|4|3|5|6|2 > 1743562 
>) — |17|4|3|5|62 + 7143562 
= |7143|562 > 1437562 

4) — |1|4)3|7562 > 1437562 
>) = |14|37562 > 4137562 

Bp = Bp = 4137562 


on vnn dD 
LY 
| 


and so &~!(7143562) = 4137562. Notice that at each stage we are reversing the 
steps of the ® algorithm, and it is easy to see this holds in general. 

An involution on a set S is a bijective map from S to S whose square is the 
identity. Foata and Schiitzenberger [20] showed that the map ibid !i, where i is the 
inverse map on permutations, is an involution on S,, that interchanges inv and maj. 

For n,k € N withO <k <n, let 


A = i -7 [n]! (1—q")(1—q""!)--(1— gt!) 


n— (=a)(1=a1)--(1=9) 
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denote the Gaussian polynomial. These are special cases of more general objects 
known as q-binomial coefficients, which are defined for x € R as 


where (a;q)x = (a)x = (1 —a)(1 — qa) --- (1 — ga) is the “q-rising factorial.” 

A partition J is a nonincreasing finite sequence A; > Az > ... of positive in- 
tegers. A; is called the ith part of A. We let €(A,) denote the number of parts, and 
|A| =¥ A; the sum of the parts. For various formulas it will be convenient to assume 
A; =0 for j > £(A). The Ferrers graph of A is an array of unit squares, called cells, 
with A; cells in the ith row, where the first cell in each row is left-justified. We of- 
ten use A to refer to its Ferrers graph. We define the conjugate partition, 2’ as the 
partition whose Ferrers graph is obtained from A by reflecting across the diagonal 
x = y, as in Figure 11.1. Here (i,j) € A refers to a cell with (column, row) coordi- 
nates (i, 7), with the lower-left-hand cell of A having coordinates (1,1). The notation 
x € A means x is a cell in A. For technical reasons we say that 0 has one partition, 
the empty set 0, with £(0) = 0 = ||. 


Figure 11.1 
On the left, the Ferrers graph of the partition (4,3,2,2), and on the right, that of its 
conjugate (4,3,2,2)’ = (4,4,2,1). 


The following result shows the Gaussian polynomials are in fact polynomials in 
q, which is not obvious from their definition. 


Theorem 11.2.3 For n,k € N, 
n+k 
aes cy) 
ACnk 


where the sum is over all partitions A whose Ferrers graph fits inside a k x n rectan- 
gle, i.e. for which A, <nand €(A) <k. 


Proof. Let 
P(nk) = YE gl 


Acnk 
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Clearly 
Ac Acn* 
Ay=n Ay<n-1 
On the other hand 


a[ntk—1) , [n—14+k) __,[n+k—-1!  (n—144)! 
q | k-1 + |= k—1! mn]! [Nn—1]! 


[k}! in]! 

= Ce tat...¢g@)+1+qt...t¢}) 
_ n+k}! 

~ Tein! 


Since P(n,0) = P(0,k) = 1, both sides of (11.1) thus satisfy the same recurrence and 
initial conditions. i 


Given a = (Q,..., a5) € N°*!, let 
{0% 1%... 5%} 


denote the multiset with a; copies of i, where Q+...+ Qs; =n. We let Mg, denote 
the set of all permutations of this multiset and refer to @ as the weight of any given 
one of these words. Also let 


denote the g-multinomial coefficient. 
The following result is due to MacMahon [26]. 


Theorem 11.2.4 Both inv and maj are multiset Mahonian, i.e., given G € Neth 


inv(o) _ n of maj(o) 
yy q a Pee > py q , he) 


ocMa ocMa 


Remark 11.2.5 Foata’s map also proves Theorem 11.2.4 bijectively. To see why, 
given a multiset permutation o of M(B) let o’ denote the standardization of o, 
defined to be the permutation obtained by replacing the Bo 0’s by the numbers | 
through Bo, in increasing order as we move left to right in o, then replacing the B; 
1’s by the numbers Bp + 1 through Bo + Bi, again in increasing order as we move left 
to right in o, etc. For example, the standardization of 31344221 is 51678342. Note 
that 


Ides(o’) C {B1,B1 + Bo,...} (11.3) 
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and in fact standardization gives a bijection between elements of M(f) and permuta- 
tions satisfying (11.3). Since the map © fixes the inverse descent set, ® maps M(B) 
to itself bijectively, sending maj to inv. 


Exercise 11.2.6 If o is a word of length n define the co-major index of 6 as follows. 


comaj(o) = y n—i. (11.4) 


Oj >O}+1 

l<i<n 
Show that Foata’s map @ implies there is a bijective map cod on words of fixed 
weight such that 


comaj(o) = inv(cod(o)). 


11.2.2 The Catalan numbers and Dyck paths 


A lattice path is a sequence of North N(0,1) and East E(1,0) steps in the first 
quadrant of the xy-plane, starting at the origin (0,0) and ending at say (m,n). We let 
Lmn denote the set of all such paths, and i the subset of Ln» consisting of paths 
that never go below the line y = “x. A iDyck path, sometimes called a Catalan 
path, is an element of L;*,, for some n. 


Let C, = ca ied) denote the nth Catalan number, so 


Co,Ci,--.=1,1,2,5,14,42,.... 


There are now over 200 known combinatorial interpretations for the Catalan num- 
bers. (See [58, Ex. 6.19, p. 219] for a list of 66 of these interpretations.) One of these 
is the number of elements of L;,,. For 1 < k <n, the number of Dyck paths from 
(0,0) to (k,k) that touch the line y = x only at (0,0) and (k,k) is Cy_1, since such 
a path must begin with a N step, end with an EF step, and never go below the line 
y =x+1 as it goes from (0,1) to (k—1,k). The number of ways to extend such a 
path to (n,n) and still remain a Dyck path is clearly C,_,. It follows that 


n 
Ga GaGa. wed (11.5) 
k=l 


There are two natural g-analogues of C,. Given 2 € Lym, let o (7) be the element 
of M (mn) resulting from the following algorithm. First initialize o to the empty string. 
Next start at (0,0), move along z and add a 0 to the end of o(2) every time a N step 
is encountered, and add a | to the end of o(7) every time an E step is encountered. 
Similarly, given 06 € M(7, 7), let (0) be the element of Ly» obtained by inverting the 
above algorithm. We call the transformation of z to o or its inverse the coding of 7 or 
o. For x € L{,,,, let a;(72) denote the number of complete squares, in the ith row from 
the bottom of , which are to the right of 2 and to the left of the line y = x. We refer 
to a;(7) as the length of the ith row of z. Furthermore call (a; (7), a2(2),...,4@,(2)) 


Catalan Paths and q,t-enumeration 687 


Figure 11.2 
A Dyck path, with row lengths on the right. The area statistic is 1+1+2+1+2=7. 


the area vector of 7, and set area(z) = ),a;(7). For example, the path in Figure 
11.2 has area vector (0,1,1,2,1,2,0), and o(7) = 00100110011101. By convention 
we say Ly, 9 contains one path, the empty path @, with area(0) = 0. 

Let M* 


(m,n 
in My, , are thus characterized by the property that in any initial segment there are at 
least as many 0’s as 1’s. The first g-analogue of C, is given by the following. 


) denote the elements o of M(,,,) corresponding to paths in ee Words 


Theorem 11.2.7 (MacMahon [54, p. 214]) We have 


y grailom) = : "| (11.6) 
TEL pn +i la 
Proof. We give a bijective proof, taken from [22]. Let Min ie Minn) \ Monn) 


and let Lim = Lnm \ Lin be the corresponding set of lattice paths. Given a path 
TELy ys let P be the point with smallest x-coordinate among those lattice points 
(i, 7) in 2 for which j —i is maximal, i.e., whose distance from the line y = x in a 
SE direction is maximal. (Since z € L,, ,, this maximal value of i — j is positive.) Let 
P’ be the lattice point on 2 before P. There must be an east step connecting P’ to P 
(preceded by another east step unless P’ is the origin). Change this east step into a 
north step and shift the remainder of the path after P up one unit and left one unit. We 
now have a path ¢(z) from (0,0) to (2—1,n+1), and moreover maj(o(¢(z))) = 
maj(o(z))—1. 

It is easy to see that this map is invertible. Given a lattice path 2’ from (0,0) 
to (n—1,n+1), let P’ be the point with maximal x-coordinate among those lattice 
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points (i, 7) in 2’ for which j —i is maximal. Thus 


maj(o) __ maj(o’)+1 _ 2n 
Eee Eee aan 
OEM, n) Oo EM (n+1,n-1) 
using (11.2). Hence 
y gril = Yo gmail) yr gnailo) (11.8) 
RELiin FEM (nn) OEM 2) 
2n 2n 1 2n 
-(7) Eel aayle ae 


The second natural g-analogue of C,, was studied by Carlitz and Riordan [15]. 
They define 


OAC me Se ae (11.10) 


TEL pn 


For example, the paths in 1a 3 given in Figure 11.3 have area, from left-to-right, 


3,2,1,1,0, soC3(q) =14+2¢4+q+¢q°. 


Figure 11.3 
The paths in L}, have Carlitz-Riordan area weights PreGacl 


Proposition 11.2.8 The recurrence relation 


Cr(q) = ¥ ag" Ck—-1Q)Cn-e(q), n> 1 (11.11) 
k=1 


holds. 


Proof. _ As in the proof of (11.5), we break up our path 7 according to the “point 
of first return” to the line y = x. If this occurs at (k,k), then the area of the part of z 
from (0, 1) to (k—1,k), when viewed as an element of L;’ , ,_,, is k—1 less than the 


area of this portion of 7 when viewed as a path in L;,. a 
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Exercise 11.2.9 Define a co-inversion of o to be a pair (i, j) withi < j and 0; < 0j. 
Show 


C,(q) = y geoim(o())—("3') (11.12) 


TEL n 


where coinv(o) is the number of co-inversions of 0. 


11.2.3. The g-Vandermonde convolution 
Let 


G1, 42, «++, Ap4+l,. = . (a1 )ko+* (Qp4i dk k 
3Gz J = 2 (11.13) 
rade ( Big dt “pee ) Gini Gi 


denote the basic hypergeometric series. A good general reference for this subject is 
[28]. The following result is known as Cauchy’s g-binomial series. 


Theorem 11.2.10 The identity 


De he ta\ a c (A)k ~ (AZ )eo 
190 & sac) ita =. Iz} <1,|q| <1 (11.14) 


holds, where (a;q)oo = (@)co = T]72.9(1 — aq’). 
Proof. The following proof is based on the proof in [28, Chap. 1]. Let 


F(a,z) = y ee 


k—0 (Qk 
Then 
F (a,z) — F(a,qz) = (1—a)zF (aq,z) (11.15) 
and 
F (a,z) — F(aq,z) = —azF (aq,z). (11.16) 


Eliminating F (aq,z) from (11.15) and (11.16) we get 


T= 
F(a,z)= : #2) p(a,q2). 
(lg) 
Iterating this n times, then taking the limit as n + oo we get 
(A254) 
F(a,z) = lim F(a, q"z 
(a,2) = lim TA" F(a,q"2) 
ae Uepy 2 eos. (11.17) 
(Z5q)oo (Z5q)eo 
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Corollary 11.2.11 (The g-binomial theorem) We have 


wre) A = (-za)a (11.18) 
k=0 
and 
= n+k k 1 
= . 11.19 
| k I: (234)n41 ) 


Proof. To prove (11.18), set a= q™" and z = —zq” in (11.14) and simplify. To 
prove (11.19), let a= q’*! in (11.14) and simplify. | 


For any function f(z), let f(z)|,« denote the coefficient of z* in the Maclaurin 
series for f(z). 


Corollary 11.2.12 The identity 

y Gea) || ME co Mere pale (11.20) 
qd k h—k = h Olas. 2 

Proof. By (11.18), 


g@) Esl = "TT a +2q*)|.4 


n—1 m— 


aay 


II 
= 
= 
+ 
NX 
OQ 
. 
Ss 
= 
= 
+ 
NX 
OQ 
= 
SQ 
ma. 
SS 
a 


k=0 j=0 


The result now reduces to the identity 


()o( st ontn-Q) atin 


a 
Corollary 11.2.13 The identity 
h 
(m+1jk {2-1 +k) |m+h—k) _ |m+n+h 
4 | i hk |= ;, (11.21) 


holds. 
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Proof. By (11.19), 


1 1 | 
Fea > left 
(z)m41 (zq™t!)n # 


> ym ea ee 
k h-k |, 
| 


We note that (11.20) and (11.21) have alternative, elementary proofs based on q- 
counting lattice paths. Both identities are special cases of the following result, known 
as the g- Vandermonde convolution. For a proof see [28, Chap. 1]. 


Theorem 11.2.14 Letn<N. Then 


261 C 1.4) = < 7 a. (11.22) 
Exercise 11.2.15 By reversing summation in (11.22), show that 
261 (* 1 aed fa) = pe (11.23) 
Exercise 11.2.16 Show Newton’s binomial series 
ye vee Yn =7 = k|<1,aeR (11.24) 


can be derived from (11.14) by replacing a by q® and letting q + \~. For simplicity 
you can assume a,z€ R. 


11.2.4 Symmetric functions 
11.2.4.1. The basics 


Given f(x1,...,%n) © K[x1,%2,.--,Xn] for some field K, and o € S,, let 
Of = f Koyis-4%a4): (11.25) 


We say f is a symmetric function if of = f for all o € S,,. It will be convenient 
to work with more general functions f depending on countably many indeterminates 
X1,X2,..., indicated by f(x1,x2,...), in which case we view f as a formal power series 
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in the x;, and say it is a symmetric function if it is invariant under any permutation 
of the variables. The standard references on this topic are [58, Chap. 7] and [53]. 
We will often let X,, and X stand for the set of variables {x1,...,x,} and {x1,x2,...}, 
respectively. 

We let A denote the ring of symmetric functions in x;,x2,... and A” the vec- 
tor subspace consisting of symmetric functions that are homogeneous of degree n. 
The most basic symmetric functions are the monomial symmetric functions, which 
depend on a partition A in addition to a set of variables. They are denoted by 
m,(X) = mz (x1,X2,.--). In a symmetric function it is typical to leave off explicit 
mention of the variables, with a set of variables being understood from context, so 
my, = mj,(X). We illustrate these first by means of examples. We let Par(n) denote 
the set of partitions of n, and use the notation A | nas an abbreviation for A € Par(n). 


Example 11.2.17 Let A = (1,1). Then we have 
m1 = \ xix; 
i<j 
=, 2, 2 2: 
ma, 11 (X3) = xpx2X3 +X1x9X3 +.41x2N3 


=x. 
i 


In general, m,(X) is the sum of all distinct monomials in the x; whose multiset of 
exponents equals the multiset of parts of A. Any element of A can be expressed 
uniquely as a linear combination of the my. 

We let 1” stand for the partition consisting of n parts of size 1. The function myn 
is called the nth elementary symmetric function, which we denote by e,. Then 


[JG +2 = Leen eg=1. (11.26) 
i=l 


Another important special case is mp, = ));x/, known as the power-sum symmetric 
functions, denoted by p,;. We also define the complete homogeneous symmetric 
functions h,, by hy, = Yj-,ma, or equivalently 


ss 
Tce) he tom. (11.27) 
i=1 _ U n=0 


For A Fn, we define e, =[];e,,. pa = Ti pa,, and ha = J]; /,. For example, 


e21= Vi xi Xj La= m2) +3m111 


| 


pai = Six Lame 


ho = hai 43a) La= m3 + 2m2,, +3111. 


i<j 


Assuming we have at least n variables, it is known that {e,, A / n} forms a basis for 
A", and so do {p,,A + n} and {hy,A bn}. 
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Definition 11.2.18 Two simple functions on partitions we will often use are 


n(4) =Y(i- Da = ) 
ZA _ J]: | 


U 


where m; = m;(A) is the number of parts of A equal to i. For example, n(54331) = 
4+6+4+9+4 = 23, and z(5433111) = 5%4%3* «2! x3! = 2160. 


Exercise 11.2.19 Use (11.26) and (11.27) to show that 


5 Den, 
A-n <A 
hy = EN. 

Aen A 


en = , 


We let @ denote the ring endomorphism @ : A — A defined by 
(pr) = (—1)*' pe. 
Thus @ is an involution with @(p,) = (—1)!4!-"A) py, and by Exercise 11.2.19, 
(en) = hy, and more generally w(e,) = hy. 
For f € A, the special value f(1,q,q7,...,q"!) is known as the principal spe- 


cialization (of order n) of f. 


Theorem 11.2.20 The following identities hold. 


en( tags) =a [OT 


m 
-1 
lig Vidyeing” “)= " el 
m 
q 
1—gi™ 
n—1 q 
Pm(1,q,---5¢" )= 1—q™ 


Proof. — The principal specializations for e,, and h,, follow directly from (11.18), 
(11.19), (11.26) and (11.27). a 


Remark 11.2.21 The principal specialization of my doesn’t have a particularly sim- 
ple description, although if ps! denotes the set of n variables, each equal to 1, then 
[58, p. 303] 


1 n 
m) (PS) a (ei et) 


where again m; is the multiplicity of the number i in the multiset of parts of 2. 
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Remark 11.2.22 A is known to be isomorphic to K[p,,p2,...]. Hence, although 
identities like 
ho) =m3 + 2m + 3m, 11 


appear at first to depend on a set of variables, it is customary to view them as polyno- 
mial identities in the py. Since the p, (in infinitely many variables) are algebraically 
independent, we can specialize them to whatever we please, forgetting about the 
original set of variables X. 


We define the Hall scalar product, a bilinear form from A x A to Q, by 


(Pa PB) =a = B), 


where for any logical statement L 


X(L) = 


1 if Lis true 
0 if Lis false. 


Clearly (f,¢) = (g, f). Also, (@f, @g) = (f,g), which follows from the definition if 
f = Pa, = pp. and by bilinearity for general f,g since the p, form a basis for A. 


Theorem 11.2.23 The hy and the mg are dual with respect to the Hall scalar prod- 
uct, 1.é., 


(ha,mg) = (A = B). (11.28) 
Proof. — See [53] or [58]. | 


For any f € A, and any basis {b,,A € Par} of A, let |p, denote the coefficient 
of b, when f is expressed in terms of the b,. Then (11.28) implies 


Corollary 11.2.24 


(fhn) =f my - (11.29) 


11.2.4.2 Tableaux and Schur functions 


Given A, © Par(n), a semi-standard Young tableau (or SSYT) of shape A and 
weight i is a filling of the cells of the Ferrers graph of A with the elements of the 
multiset {14'242-.--}, so that the numbers weakly increase across rows and strictly 
increase up columns. Let SSYT(A,) denote the set of these fillings, and Ky ,, the 
cardinality of this set. The Kj ,, are known as the Kostka numbers. Our definition 
also makes sense if our weight is a weak composition of 7 , i.e., any finite sequence 
of nonnegative integers whose sum is n. For example, K(3,2) (22,1) = K(3,2),(2,1,2) = 
K3.2),(1,2,2) = 2 as in Figure 11.4. 

If the Ferrers graph of a partition f is contained in the Ferrers graph of A, denoted 
B CA, let A/B refer to the subset of cells of A that are not in B. This is referred to 
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2|2 ale 
fase fee |e 
2|3 3/3 
Ee Peeae: 
2|3 3| 3 
SS Nel a 


Figure 11.4 
Some SSYT of shape (3,2). 


as a Skew shape. Define a SSYT of shape A/B and weight v, where |v| = |A|—|f], 
to be a filling of the cells of A/B with elements of {1”'2”2---}, again with weak 
increase across rows and strict increase up columns. The number of such tableaux is 
denoted K} /g y. 

Let wcomp() denote the set of all weak compositions whose multiset of nonzero 
parts equals the multiset of parts of wu. It follows easily from Figure 11.4 that 
K(3.2),0 = 2 for all @ € wcomp(2,2, 1). Hence 


DTDs" =2me2,1; (11.30) 


where the sum is over all tableaux T of shape (3,2) and weight some element of 
weomp(2,2, 1). 
This is a special case of a more general phenomenon. For A € Par(n), define 


a=), 


aT i 


where the sum is over all weak compositions @ of n, and all possible tableaux T of 
shape A and weight a. Then 


sy = VK ym, (11.31) 
brn 


i.e., Ky g = Ky .q for all compositions B,a whose multiset of parts is the same (we 
leave it to the interested reader to prove this fact bijectively). The s, are called Schur 
functions, and are fundamental to the theory of symmetric functions. Two special 
cases of (11.31) are sy = hy (since Kn, = 1 for all pp € Par(n)) and sin = ey (since 
Kin =X(U= 1"). 


696 Handbook of Enumerative Combinatorics 


A SSYT of weight 1” is called standard, or a SYT. The set of SYT of shape A 
is denoted SYT(A). For (i,j) € A, let the content of (i, 7), denoted c(i, j), be i— j. 
Also, let h(i, 7) denote the hook length of (i, j), defined as the number of cells to the 
right of (i,j) in row j plus the number of cells above (i,j) in column i plus 1. For 
example, if A = (5,5,3,3, 1), 4(2,2) =6. It is customary to let f* denote the number 
of SYT of shape A, i.e., sie = Ky jn. There is a beautiful formula for hig namely 


n! 
Tajjea ta) 


Below we list some of the important properties of Schur functions. See [58, Chap- 
ter 7] for proofs of these well-known identities, and how (11.32) can be derived from 
(11.33). 


fl= (11.32) 


Theorem 11.2.25 Let A, u € Par. Then 


I. The Schur functions are orthonormal with respect to the Hall scalar product, 
Lé., 


<54,5p >=1(A =H). 


Thus for any f € A, 
<fi5. >= fs. 
2. Action by @: 
(54) = sy. 
3. Principal Specialization: For A € Par, 


_ Inta—]] 
sa(1,4,97,--.,4" yg Il tT. 177491? (11.33) 
ager ett + 


where for a given square (i,j) € A, we define the coarm a’ and coleg I' as in 
Figure 11.5 from Section 11.3. 


4. Cauchy Identities: For any two alphabets of variables X,Y, let XY denote the 
set of variables {x;y ;}. Then 


en(XY)= Yo sq(X)sqr(¥) (11.34) 
A€Par(n) 


hy(XY)= )i sa(X)sq(¥). 
A€Par(n) 
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11.2.4.3 Statistics on tableaux 


There is a q-analogue of the Kostka numbers, denoted by Ky , (q), which has many 
applications in representation theory and the combinatorics of tableaux. Originally 
defined algebraically in an indirect fashion, the Kj ,,(q) are polynomials in q that sat- 
isfy Kq (1) = Ky y. Foulkes [21] conjectured that there should be a statistic stat(T ) 
on SSYT T of shape A and weight f such that 


Kip (q) = y Ga), 
TESSYT (A) 


This conjecture was resolved by Lascoux and Schiitzenberger [45], who found a 
statistic charge to generate these polynomials. Butler [14] provided a detailed ac- 
count of their proof, filling in a lot of missing details. A short proof, based on the 
combinatorial formula for Macdonald polynomials, is contained in [39, Appendix 
A]. 

Assume we have a tableau T € SSYT(A, 1) where wu € Par. It will be more con- 
venient for us to describe a slight modification of charge(T), called cocharge(T), 
which is defined as n(u) — charge. The reading word read(T) of T is obtained by 
reading the entries in T from left to right in the top row of 7, then continuing left to 
right in the second row from the top of T, etc. For example, the tableau in the upper- 
left of Figure 11.4 has reading word 22113. To calculate cocharge(T), perform the 
following algorithm on read(T). (Note: The algorithm below applies to any word, 
not just words that are the reading word of some SYT.) 


Algorithm 11.2.26 


1. Start at the end of read(T) and scan left until you encounter a 1, say this 

occurs at spot i}, so read(T);, = 1. Then start there and scan left until you 
encounter a 2. If you hit the end of read(T) before finding a 2, loop around 
and continue searching left, starting at the end of read(T). Say the first 2 you 
find equals read(T);,. Now iterate, start at in and search left until you find a 3, 
etc. Continue in this way until you have found 4,5,...,[1, with Uy, occurring 
at spot iy,. Then the first subword of textread(T ) is defined to be the elements 
of the set {read(T );,,...,read(T )i,,, }, listed in the order in which they occur 
in read(T) if we start at the beginning of read(T) and move left to right. For 
example, if read(T) = 21613244153 then the first subword equals 632415, 
corresponding to places 3,5,6,8,9, 10 of read(T). 
Next remove the elements of the first subword from read(T ) and find the first 
subword of what’s left. Call this the second subword. Remove this and find the 
first subword in what’s left and call this the third subword of read(T), etc. For 
the word 21613244153, the subwords are 632415, 2143, 1. 


2. The value of charge(T ) will be the sum of the values of charge on each of the 
subwords of rw(T ). Thus it suffices to assume rw(T ) € Sin for some m, in which 
case we set 


cocharge(rw(T)) = comaj(rw(T)~!), 
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where read(T)~! is the usual inverse in S,,, with comaj as in (11.4). (Another 
way of describing cocharge(read(T )) is the sum of m—i over all i for which 
i+1 occurs before i in read(T).) For example, if 6 = 632415, then o~! = 
532461 and cocharge(o) =5+4+1 = 10, and finally 


cocharge(21613244153) = 10+4+0= 14. 


Note that to compute charge, we could create subwords in the same manner, and 
count m—i for each i with i+ 1 occurring to the right of i instead of to the left. For 
A, € Par(n) we set 


Ry u(q) =F") Ky w(1/a) 
gore) ; 


TESSYT (AL) 


In addition to the cocharge statistic, there is a major index statistic on SYT that 
is often useful. Given a SYT T of shape A, define a descent of T to be a value of i, 
1 <i<|A|, for which i+ 1 occurs in a row above iin T. Let 


maj(T) = yi 
comaj(T) = y |A| —i, 
where the sums are over the descents of T. Then [58, p.363] 


1 maj 
Lads) = 7 y gam 


11.2.5 Representation theory 


Let G be a finite group. A (matrix) representation of G is a group homomorphism 
from G to GL,,(C), the set of invertible n x n matrices with entries in C. See [56] and 
[44] for a detailed discussion of the representation theory of the symmetric group 
and other finite groups. We include an informal discussion of some of the main ideas 
here, in order to motivate the combinatorial problems we will be discussing. 

We will identify a representation with the image of a homomorphism from G 
to GL,(C), namely the set of square invertible matrices {M(g), g € G} with the 
property that 


M(g)M(h) =M(gh) for all g,h EG. (11.35) 


On the left-hand side of (11.35) we are using ordinary matrix multiplication, and on 
the right-hand side, to define gh, multiplication in G. The number of rows of a given 
M(g) is called the dimension of the representation. 
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An action of G on a set S is a map from G x S to S, denoted by g(s) for g € G, 
s € S, which satisfies 


g(h(s)) =(gh)(s) forall g,heEG,s ES, 


with e(s) = s for all s € S, where e is the identity in G. Let V be a finite-dimensional 
C-vector space, with basis w1,w2,...Wn. Any linear action of G on V makes V into 
a CG module. A module is called irreducible if it has no submodules other than 
{0} and itself. Maschke’s theorem [44] says that every nonzero CG-module V can 
be expressed as a direct sum of irreducible submodules. 

If we form a matrix M(g) whose ith row consists of the coefficients of the w; 
when expanding g(w;) in the w basis, then {M(g), g € G} is a representation of G. 
In general {M(g),g € G} will depend on the choice of basis, but the trace of the 
matrices will not. The trace of the matrix M(g) is called the character of the module 
(under the given action), which we denote char(V). If V = Oi , Vj, where each V; 
is irreducible, then an ordered basis of V can be chosen so that the matrix M will be 
in block-diagonal form, where the sizes of the blocks are the dimensions of the Vj. 
Clearly char(V) = Yes char(Vj;). It turns out that there are only a certain number of 
possible functions that occur as characters of irreducible modules, namely one for 
each conjugacy class of G. These are called the irreducible characters of G. 

In the case G = Sy, the conjugacy classes are in one-to-one correspondence with 
partitions A € Par(n), and the irreducible characters are denoted x”. The dimension 
of a given V, with character xe is known to be ee The value of a given x’ (6) 
depends only on the conjugacy class of o. For the symmetric group the conjugacy 
class of an element is determined by rearranging the lengths of the disjoint cycles of 
o into nonincreasing order to form a partition called the cycle type of o. Thus we 
can talk about ya (B), which means the value of yh at any permutation of cycle type 
B. For example, 7)(B) = 1 for all B tn, so x) is called the trivial character. 
Also, 7!"(B) = (—1)"—“) for all B En, so y'" is called the sign character, since 
(—1)"-“(B) is the sign of any permutation of cycle type B. 

One reason Schur functions are important in representation theory is the follow- 
ing (see [58, p. 347] and [53, Chapter 1]). 


Theorem 11.2.27 When expanding the sj into the p, basis, the coefficients are the 
x’. To be exact 


Pu= Vi x*(u)sa 
AEn 


= Vo zx" (pw 
ubn 


Let C[X,,] = C[x1,...,x,]. Given f(x1,.-.,Xn) € C[X,] and o € S,, then 


Of = f(Xo,5-++ Xo, ) 


defines an action of S,, on C[X,]. 


700 Handbook of Enumerative Combinatorics 


Assume V is a homogeneous subspace of C[X,,] which can be decomposed as 


where V“) is the subspace consisting of all elements of V that are homogeneous of 
degree i in the x;, and is finite-dimensional. This gives a grading of the space V, and 
we define the Hilbert series #(V;q) of V to be the sum 


H(V3q) = Ygidim(v"), 
i=0 


where dim indicates the dimension as a C-vector space. If in addition V is fixed by 
the S,, action, we define the Frobenius series 7 (V;q) of V to be the symmetric 
function 


Ya YY Mult(y7,v)s,, 
i=0  A€Par(i) 


where Mult( x% , vi) is the multiplicity of the irreducible character x* in the charac- 
ter of V under the action. In other words, if we decompose V into irreducible Sn- 
submodules, Mult(z2,V) is the number of these submodules whose trace equals 


Aa 
x”. 
A polynomial in C[X,,] is alternating, or an alternant, if 


of =(-1)™O Ff forall o € Sp. 
The set of alternants in V forms a subspace called the subspace of alternants, or 
anti-symmetric elements, denoted V®. This is also an S,,-submodule of V. 


Proposition 11.2.28 The Hilbert series of V& equals the coefficient of s\n in the 
Frobenius series of V, i.e., 


HE (V*3q) = (F(V3q),51") - 


Proof. Let B be a basis for V with the property that the matrices M(o) are in 
block form. Then b € B is also in (V®) if and only if the column of M(o) corre- 
sponding to b has entries (—1)!"(°) on the diagonal and 0’s elsewhere, i.e., is a block 
corresponding to 7!". Thus 


(F (V3q),81") = ¥ gidim((v')) = H(Vsq). 
i=0 
Z 


Remark 11.2.29 Since the dimension of the representation corresponding to + 
equals va which by (11.29) equals < sy ,hy» >, we have 


(F(V3q),hin) = A(V3q). 
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Example 11.2.30 Since a basis for C[X,] can be obtained by taking all possible 
monomials in the x;, 


HE (C[Xn)3q) = (1-4) 
Taking into account the S,-action, it is known [40, Section 1.4] that 


aj(T) 


ys Yresyr(a)q" 


F (C[Xn]3) = a a) (11.36) 
A€Par(n) q)n 
= Yo asa(la¢,. =[I;—==5 
A€Par(n) ij (1 ~ghag)| 


11.2.5.1 The ring of coinvariants and the space of diagonal harmonics 


The set of symmetric polynomials in the x;, denoted C[X,|*", which is generated by 
1,e1,...€,, is called the ring of invariants. The quotient ring R, = C[x,...,%n]/< 
€1,€2,---,€n >, or equivalently C[x1,...,X%n]/ < pi,p2,---;Pn >, obtained by form- 
ing the quotient by the ideal generated by all symmetric polynomials of positive de- 
gree, is known as the ring of coinvariants. It is known that R,, is finite dimensional 
as a C-vector space, with dim(R,,) =n!, and more generally that 


FC (Rn3q) = [n!. 
E. Artin [10] derived a specific basis for R,, namely the cosets of 


{ [] x°,0< a; <i-1J}. 


1<i<n 


Also, 


FRug= Ys YY gra”, (11.37) 
AePar(n) TESYT(A) 


a result that Stanley [57, 59] attributes to unpublished work of Lusztig. This shows 
the Frobenius series of R, is (¢), times the Frobenius series of C[X,]. 
Let 


n—1 

1 x . ie 

Wo Xe 48 7 
2 

waat} PO P= TY (yaa 
: 1<i<j<n 

-1 

1 Xp oo 


be the Vandermonde determinant. The space of harmonics H,, can be defined as the 
C-vector space spanned by V,, and its partial derivatives of all orders. Haiman [40] 
provides a detailed proof that H,, is isomorphic to R, as an S, module, and notes 
that an explicit isomorphism @ is obtained by letting @(h),h € Hy, be the element of 
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C[X,,] represented modulo < e1,...,@, > by h. Thus dim(H,,) =n! and moreover the 
character of H,, under the S,,-action is given by (11.37). He also argues that (11.37) 
follows immediately from (11.36) and the fact that H,, generates C[X,] as a free 
module over C[X;,]°" 

There is a natural extension of this construction to two sets of variables, which 
has a very rich algebraic and combinatorial structure. Let the ring of diagonal coin- 
variants DR,, be defined as 


DRn = C[Xn5 Yn] tes sivhek 0), 


By analogy we also define the space of diagonal harmonics DH,, by 
n 
DH, = 6 f € C[Xn, Yn] 1 BC Ax;)"(dy;)* f =0,Vh+k>0 


Many of the properties of H,, and R, carry over to two sets of variables. For ex- 
ample the symmetric group acts “diagonally” on DR,, and DH, by permuting the X 
and Y variables in the same way, which turns DH,, and DR,, into finite-dimensional 
isomorphic S,-modules. We can decompose DH, by homogeneous X and Y degree, 
and the S,, action respects this bi-grading. Hence we can talk about the Hilbert series 
JH (DH) ;q,t) and and the Frobenius series .¥ (DH; q,t). 


11.3. The g,t-Catalan numbers 


Given a cell x € A, let the arm a = a(x), leg / = I(x), coarm a’ = a’ (x), and coleg 
I’ = I'(x) be the number of cells strictly between x and the border of A in the east, 
north, west, and south directions, respectively, as in Figure 11.5. Also, define 


/ 


Bu SJ gt, t! My =My(9,t) =] [U-4*0’), 


xEM XxEU 
where a prime symbol’ above a product or a sum over cells of a partition Lt indicates 
we ignore the corner (1,1) cell, and Bg = 0, Tg = 1. For example, Boga) =1t+¢qt 
t+qt+0? and Tio2,1) = (1—g)(1 —1)(1 — qt) (1 —#°). Note that 


= Ser 


xEU xELL 


In 1996 Garsia and Haiman [27] introduced an amazing two-parameter Catalan 
sequence, C,,(q,t), which they defined as the following sum of rational functions: 


T2M11,B 
Ca(gt)= y=, (11.38) 


Ln Wu 
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Figure 11.5 
The arm a, coarm a’, leg 1, and coleg I’ of a cell. 


M=(1-g)0-), Hag), we=TT@-! ye -g"). 
xEUL 


The definition of C,(q,t) was motivated by ideas involving algebraic geometry, and 
Garsia and Haiman conjectured that C,(q,t) € N{g,t]. More specifically, they con- 
jectured that C,,(q,t) is the sign character in the Frobenius series for DHy, i.e., 


Ci(q,t) = (F(DHn3q,t), 81"). (11.39) 


Mark Haiman proved this conjecture in 2001 by obtaining the following expression 
for ¥ (DH,;q,t) in terms of Macdonald polynomials. 


Theorem 11.3.1 [42] We have 
TyMMy Bu Hy (X3q,t 
F (DEy;q,t) = Y == eae (11.40) 
burn Wu 


where the H, u(X;q,t) form the modified Macdonald polynomial symmetric function 
basis. 


Although we will not describe these polynomials explicitly here, the interested reader 
can find a combinatorial description of them in [36] or [39, Appendix A]. 
We mention that 


(Ay (X3q,t),51»(X)) =~ Th, 


so (11.40) implies (11.39). The right-hand side of (11.40) can be expressed more 
compactly as Ve,(X), where V is the linear operator on symmetric functions satisfy- 
ing 


VA (X;q,t) = Ty Hy (X34q,t). 


Hence we can refer to ¥(DH,;q,t) and Ve, interchangeably. 
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Around the same time Haiman proved Theorem 11.3.1, Garsia and Haglund 
proved independently that C,(q,t) can be expressed combinatorially in terms of 
statistics on Dyck paths which we now describe. 


11.3.1 The bounce statistic 


Our combinatorial formula for C,(g,t), the g,t-Catalan number, involves a new 
statistic on Dyck paths we call bounce. 


Definition 11.3.2 Given 1 € Ly, 
scribed by the following algorithm. 

Start at (0,0) and travel North along m until you encounter the beginning of an 
E step. Then turn East and travel straight until you hit the diagonal y = x. Then turn 
North and travel straight until you again encounter the beginning of an E step of %, 
then turn East and travel to the diagonal, etc. Continue in this way until you arrive 
at (n,n). 

We can think of our bounce path as describing the trail of a billiard ball shot 
North from (0,0), which “bounces” right whenever it encounters a horizontal step 
and “bounces” up when it encounters the line y = x. The bouncing ball will strike 
the diagonal at places 


define the bounce path of 7 to be the path de- 


(0,0) (jis Jt); JosJa)secas | eat yb=1) sei db) = (n,n). 


We define the bounce statistic bounce(1) to be the sum 


bounce(m) = y n— ji, 
i=1 


and we call b the number of bounces, with j, the length of the first bounce, j2— j, the 
length of the second bounce, etc. The lattice points where the bouncing billiard ball 
switches from traveling North to East are called the peaks of 1. The first peak is the 
peak with smallest y coordinate, the second peak the one with next smallest y coordi- 
nate, etc. For the path 7 in Figure 11.6, there are five bounces of lengths 3,2,2,3,1 
and bounce(1) = 19. The first two peaks have coordinates (0,3) and (3,5). 


Let 


(q,t ae gen ) gbounce(z) 
Fil; 


mELihn 
Theorem 11.3.3 The equality 
Ci(q,t) = Fi(q,t) (11.41) 
holds. 


Combining Theorem 11.3.3 with Theorem 11.3.1 we have the following. 
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Figure 11.6 
The bounce path (dotted line) of a Dyck path (solid line). The bounce statistic equals 
11—3+11—54+11—74+11—-10=84+6+44+1=19. 


Corollary 11.3.4 The identity 


H(DH*:4q,t) = y grainy poker) (11.42) 


TEL n 
holds. 


Theorem 11.3.3 was first conjectured by Haglund in 2000 [33] after a prolonged 
study of tables of C,(q,t). It was then proved by Garsia and Haglund [25, 26]. At the 
present time there is no known way of proving Corollary 11.3.4 without using both 
Theorems 11.3.3 and 11.3.1. 

The proof of Theorem 11.3.3 is based on a recursive structure underlying F;,(q,t). 


Definition 11.3.5 Let Lit,,(k) denote the set of all 1 € L;.,, which begin with exactly 
KN steps followed by an E step. By convention Leo (k) consists of the empty path if 


k = 0 and is empty otherwise. Set 


Fix(q,t) = 3 gira pomeay. Fro = x(n = 0). 
TEL n(k) 
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Theorem 11.3.6 [33]. For1<k<n, 


" [rtk—-1) ng ( 
Fux at) = | . 1Kg(2) Fe (gst). (11.43) 
r=0 qd 


Proof. Given B € L;*,,(k), with first bounce k and second bounce say r, then B must 
pass through the lattice points with coordinates (1,k) and (k,k-+r) (the two large dots 
in Figure 11.7). Decompose B into two parts, the first part being the portion of B 
starting at (0,0) and ending at (k,k+-r), and the second the portion starting at (k,k + 
r) and ending at (n,n). If we adjoin a sequence of r N steps to the beginning of the 
second part, we obtain a path B’ in Ler Paci Th It is easy to check that bounce(B) = 
bounce(B’) +n —k. It remains to relate area($) and area(B’). 

Clearly the area inside the triangle below the first bounce step is ar If we fix B’, 
and let B vary over all paths in L;*,,(k) that travel through (1,k) and (k,k +r), then 
the sum of grralB ) will equal 


gr8g®) rae ‘| 
q 


r 


by (11.1). Thus 


n—k 
Fy 4(q,t) = y y giteal(B!) ,bounce(B') jk g(2) ie i 
| OB ET pai) q 
n—k 
r+k—1) 4 4 (k 
> ae a cee 
r=0 qd 
a 
Corollary 11.3.7 We haves 
: oy +203+...4(b— lay ,Y? (%) FA Oj + Oi41—1 
F,(q,t) = y 2 Be bqhi=l\2 ; 
b=10,4+004+...4+Qp=n i=1 Qi+1 q 
a;>0 
(11.44) 


where the inner sum is over all compositions of n into b positive integers. 


Proof. — This follows by iterating the recurrence in Theorem 11.3.6. The inner term 
in the sum over b equals the sum of g2"°#r5°""°e over all paths 2 whose bounce path 
has b steps of lengths a),...,Q,. For such a 7, the contribution of the first bounce 
to bounce(z) is n— Q = OQ) +...+, the contribution of the second bounce is 
nN— Ol, — A = 03+...+ Q%, et cetera, so bounce(7) = Q) +203+...+(b—1)o. 0 
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Figure 11.7 
A path whose first two bounce steps are k and r. 


11.3.2 The special values ¢ = 1 and t = 1/q 


Garsia and Haiman proved that 
Cn(q,1) = Cr(q) 


ewig =|" 11.45 
q\)Cr(q; 1/4) erie], (11.45) 
which shows that both the Carlitz-Riordan and MacMahon q-Catalan numbers are 
special cases of C,(q,t). In this section we derive analogous results for F,,;(g, 1) and 
Frk(qs 1/q). 

By definition we have 


Fn(q.1) =Cn(q)- 


It is perhaps worth mentioning that the F,,.(q, 1) satisfy the simple recurrence 


n 
Fag(a:l) = YQ? Fn-19-1(@, 1) Fi n(4, 1): 


m=k 


This follows by grouping paths in Lin (k) according to the first time they return to 


the diagonal, at say (m,m), then arguing as in the proof of Proposition 11.11. 
The F,,.(q, 1/q) satisfy the following refinement of (11.45). 
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Theorem 11.3.8 For 1<k<n, 


gq?) Fnx(q,1/q) = m | 


Proof, Since Fyn(q,t) = q), Theorem 11.3.8 holds for k =n. If 1 <k <n, we 
start with Theorem 11.3.6 and then use induction on 1: 


n—k 


q) Fra(qsq') eo g2)q-( 2) y qe?) Fy-erlaaq7)q@)--9 es ' 
q 


r 


1 
q q 


gen ies ‘1 [r] pee ee (es ae 


= r [In—k] | n—k—-1-(r-1) 
— ,(k-1)n [k] ae, k+u u(n—k) 2(n—k)—2—u 


Using (11.19) we can write the right-hand side of (11.46) as 


(e—-1)n_[K] 1 1 
[n —k] (zq"* itt (Z)n—k 


[n—k] n 
a 
Corollary 11.3.9 The identity 
q°) Fx(q.1/q) = =a aN 
holds. 
Proof. _N. Loehr has pointed out that we can use 
Frvii(qt) =t"Fn(q,t), (11.47) 


which by Theorem 11.3.8 implies 


nl — [lt] [2@@4+1)-2] — GA) [2n 
a! Vnaatatia) = | nt+1-1 | =e, 
= q"2)-"F,(q,1/4) = 4 Fa(g, 1/4): 
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11.3.3. The symmetry problem and the dinv statistic 


From its definition, it is easy to show C,(q,t) =C,(t,q), since the arm and leg values 
for equal the leg and arm values for 1’, respectively, which implies g and f are 
interchanged when comparing terms in (11.38) corresponding to ps and pt’. This also 
follows from the theorem that C,(q,t) = # (DH; ;q,t). Thus we have 


y gre) gbouncele) = y genta) area (11.48) 


TEL n TEL n 


a surprising statement in view of the apparent dissimilarity of the area and bounce 
statistics. At present there is no other known way to prove (11.48) other than as a 
corollary of Theorem 11.3.3. 


Problem 11.3.10 Prove (11.48) by exhibiting a bijection on Dyck paths that inter- 
changes area and bounce. 


A solution to Problem 11.3.10 should lead to a deeper understanding of the com- 
binatorics of DH,. We now give a combinatorial proof from [33] of a very special 
case of (11.48), by showing that the marginal distributions of area and bounce are 
the same, i-e., F,(g,1) = F,(1,q). 


Theorem 11.3.11 The identity 


y qialr) = y qhounce(x) (11.49) 


TEL n TEL y 


holds. 


Proof. Given m7 € Lie let aja2---d, denote the sequence whose ith element is 
the ith coordinate of the area vector of 7, i.e., the length of the ith row (from the 
bottom) of z. A moment’s thought shows that such a sequence is characterized by 
the property that it begins with zero, consists of n nonnegative integers, and has no 
2-ascents, i.e., values of i for which aj,; > a;+ 1. To construct such a sequence 
we begin with an arbitrary multiset of row lengths, say {0%1%---(b—1)%} and 
then choose a multiset permutation t of {0% 1%} that begins with 0 in (“! ee) 
ways. Next we will insert the a3 twos into T, the requirement of having no 2-ascents 
translating into having no consecutive 02 pairs. This means the number of ways to 


do this is (2 eves independent of the choice of t. The formula 


me b Ge p=! a, —1+4; 
F@l=y Yo gna "TT ( i) (11.50) 


b=10+...40,=n i=l +1 
aj>0 
follows, since the product above counts the number of Dyck paths with a specified 
multiset of row lengths, and the power of g is the common value of area for all these 
paths. Comparing (11.50) with the g = 1,t = q case of formula (11.44) completes the 
proof. | 
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There is another pair of statistics for the g,t-Catalan discovered by M. Haiman 
[41]. It involves pairing area with a different statistic we call dinv, for diagonal 
inversion or d-inversion. It is defined, with a; the length of the ith row from the 
bottom, as follows. 


Definition 11.3.12 Let x € Lj. Let 


din(x) =|{(i,j):1<i<j<n a;=a;}| 
+HGfil<i<j<n aj=aj+l}l. 


In words, dinv(%) is the number of pairs of rows of 1 of the same length, or that 
differ by one in length, with the longer row below the shorter. For example, for the 
path on the left in Figure 11.8, with row lengths on the right, the inversion pairs (i, j) 
are (3,7),(4,7),(5,7),(6,8) (corresponding to rows that differ by one in length), 
and (2,7), (3,4), (3,5), (3,8), (4,5), (4,8), (5,8) (corresponding to pairs of rows of 
the same length), thus dinv = 11. We call inversion pairs between rows of the same 
length “equal-length” inversions, and the other kind “offset-length” inversions. 


Figure 11.8 
A path z with row lengths to the right, and the image ¢ (71). 


Theorem 11.3.13 The identity 
y gn ree) = y gr ores) (11.51) 


TEL MEL 


holds. 


Proof. | We will describe a bijective map ¢ on Dyck paths with the property that 


dinv(z) = area(¢(z)) (11.52) 
area(7) = bounce(¢(z)). 
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Say b— 1 is the length of the longest row of 7. The lengths of the bounce steps of € 
will be ,...,Q,, where Qj; is the number of rows of length i— 1 in 7. To construct 
the actual path C, place a pen at the lattice point (1, 0 + Q2) (the second peak of 
the bounce path of €). Start at the end of the area sequence and travel left. Whenever 
you encounter a 1, trace a South step with your pen. Whenever you encounter a 0, 
trace a West step. Skip over all other numbers. Your pen will end up at the top of the 
first peak of €. Now go back to the end of the area sequence, and place your pen at 
the top of the third peak. Traverse the area sequence again from right to left, but this 
time whenever you encounter a 2 trace out a South step, and whenever you encounter 
a 1, trace out a West step. Skip over any other numbers. Your pen will end up at the 
top of the second peak of €. Continue at the top of the fourth peak looking at how 
the rows of length 3 and 2 are interleaved, etc.. See Figure 11.8. 

It is easy to see this map is a bijection, since given ¢, from the bounce path we can 
determine the multiset of row lengths of 7. We can then build up the area sequence 
of 7 just as in the proof of Theorem 11.3.11. From the portion of the path between 
the first and second peaks we can see how to interleave the rows of lengths 0 and 1, 
and then we can insert the rows of length 2 into the area sequence, etc. 

Note that when tracing out the part of ¢ between the first and second peaks, 
whenever we encounter a 0 and trace out a West step, the number of area squares 
directly below this West step and above the bounce path of € equals the number of 
1’s to the left of this 0 in the area sequence, which is the number of offset-length 
inversion pairs involving the corresponding row of length 0. Since the area below 
the bounce path clearly counts the total number of equal-length inversions, it follows 
that dinv(z) = area(¢(z)). 

Now by direct calculation, 


bounce(€) =n— a, +n— 0, —O)4+...n—-Q—...— Ap_} (11.53) 


b-1 
= (Q)+...+ 0) +...+(a%) = ¥ io, =area(z). 
i=l 


Remark 11.3.14 The construction of the bounce path for a Dyck path occurs in 
an independent context, in work of Andrews, Krattenthaler, Orsina and Papi [5] on 
the enumeration of ad-nilpotent ideals of a Borel subalgebra of sl(n + 1,C). They 
prove the number of times a given nilpotent ideal needs to be bracketed with itself to 
become zero equals the number of bounces of the bounce path of a certain Dyck path 
associated to the ideal. Another of their results is a bijective map on Dyck paths that 
sends a path with b bounces to a path whose longest row is of length b—1. The € 
map above is just the inverse of their map. Because they only considered the number 
of bounces, and not the bounce statistic per se, they did not notice any connection 
between C,,(q,t) and their construction. 


Theorem 11.3.3 now implies the following. 
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Corollary 11.3.15 We have 


Cn(q,t) & y greene) (11.54) 
TEL n 
We also have 
Fix (q,t) = y gin yarn: (11.55) 
mELy , 


1 has exactly k rows of length 0 


since under the € map, paths with k rows of length 0 correspond to paths whose first 
bounce step is of length k. 


Remark 11.3.16 N. Loehr has noted that if one can find a map that fixes area and 
sends bounce to dinv, by combining this with the € map one would have a map that 
interchanges area and dinv, solving Problem 11.3.10. 


Exercise 11.3.17 Let 
Gnx(q,t) = om HL dene (11.56) 


mel, 
1 has exactly k rows of length 0 


Without referencing any results on the bounce statistic, prove combinatorially that 


ny MOF pt k—1 
Gnx(q,t) = p-kq(s) y | e Gy_xr(q,t). 
r=0 qd 


Exercise 11.3.18 Haiman’s conjectured statistics for C,(q,t) actually involved a 
different description of dinv. Let A(x) denote the partition consisting of the (5) — 
area() squares above % but inside the n x n square. (This is the Ferrers graph of a 
partition in the so-called English convention, which is obtained from the French con- 
vention of Figure 11.1 by reflecting the graph about the x-axis. In this convention, the 
leg I(s) of a square s is defined as the number of squares of 1 below s in the column 
and above the lower border m, and the arm a(s) as the number of squares of A to 
the right and in the row.) Then Haiman’s original version of dinv was the number of 
cells s of A for which 


I(s) <a(s) <U(s) +1. 


Prove this definition of dinv is equivalent to Definition 11.3.12. 


11.3.4 q-Lagrange inversion 


q-Lagrange inversion is useful when analyzing the special case t = | of ¥(DH);4q,t). 
In this section we derive a general g-Lagrange inversion theorem based on work of 
Garsia and Haiman. We will be working in the ring of formal power series, and we 
begin with a result of Garsia [23]. 
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Theorem 11.3.19 /f 
(F 0qgG)(z) = fnG(z)G(qz)--G(q""'2), 
n 


where F =, fnz", then 
FogG=zandGo,1F =z 
are equivalent to each other and also to 
(bo, 1F)ogG=P=(PoyG)o, iF forall ®. 


Given z € Ly, let B(2) = Bi (7) Bo(x)--- denote the partition consisting of the 
vertical step lengths of 7 (i.e., the lengths of the maximal blocks of consecutive 0’s 
in o(7)) arranged in nonincreasing order. For example, for the path on the left in 
Figure 11.8 we have B = (3,2,2,1). By convention we set B (0) =. Define H(z) via 
the equation 1/H(—z) := Li exz*. Using Theorem 11.3.19, Haiman [40, pp. 47-48] 
derived the following. 


Theorem 11.3.20 There is a unique solution h*(q),n > 0 to the equation 


© co 


Vie’ = Vg Oarga(-q aH (-q22)--H(-""2), (gq) = 1. 
k=0 n=0 


Forn > 0, h2(q) has the explicit expression 
a= Y gf *egig). 
NEL} n 
For example, we have 
Ws(q) = qre3 +q°e2,1 + 2ge2,1 +e,3. 
We now derive a slight generalization of Theorem 11.3.20 that stratifies Dyck 
paths according to the length of their first bounce step. 


Theorem 11.3.21 Let cy,k > 0 be a set of variables. Define h}(c,q),n > 0 via the 
equation 


co 


vec =) gq hi (c,q)z"H(—q7!z)H(—q-2z)---H(-4-"2), (e.g) =e. 
k=0 n=0 

(11.57) 
Then for n > 0, hi(e,q) has the explicit expression 


n 


Reg=Yicc Yo Megiq). 
k=0 EL n(k) 


For example, we have 


h3(e,q) = qre3c3 + (q7e2,1 +. ge2,1)c2 + (ge2,1 +13) C1. 
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Proof. Our proof follows Haiman’s proof of Theorem 11.3.20 closely. Set 
A*(z,€3q) = Vpeohn(e.9)z", A" (zg) = Un-oln(g)e", ® = H*(zq,06:9), F = 
zH(—z), and G = zH*(qz;q). Replacing z by zq in (11.57) we see that Theorem 
11.3.21 is equivalent to the statement 


3 exceghk = = 3 q" hi, (c,q)zH (—z)zq7'H(—q7'z)-+-zq'"H(—q'"z) 
k=0 n=0 
=> Po,-1 F. 


On the other hand, Theorem 11.3.20 can be expressed as 


= Yi g Ont (gheH(—z/4) --H(—2/4"), 


r= La Dns (q)e"* 1H (2) (2/4) --H(—2/q") 
= Fania) eHt(-2)}{ 2a(-z/a) bf Saale} 


=Go,1F. 
Thus assuming Theorem 11.3.21 we have, using Theorem 11.3.19, 
® = (Po,1F)ogG (11.58) 
= (i exteg'2") 0qG. 


Comparing coefficients of z” in (11.58) and simplifying we see that Theorem 11.3.21 
is equivalent to the statement 


nk ko. A 
g'his(e,q) = Yiq@)**ece = gt *T gi Pink (q). (11.59) 
k=0 ny t...+np=n—k i=1 
nj>0 


To prove (11.59) we use the “factorization of Dyck paths” as discussed in [40]. 
This can be be represented pictorially as in Figure 11.9. The terms multiplied by 
cx correspond to z € L;*,(k). The area of the parallelogram whose left border is on 
the line y = x+i—1 is (i—1)n;, where n, is the length of the left border of the 
parallelogram. Using the fact that the path to the left of this parallelogram is in Ly ror 
(11.59) now becomes transparent. 


1 


Letting e, = 1, cj = X(j =k) and replacing q by q-* and z by z/g in Theorem 


11.3.21 we get the following. 
Corollary 11.3.22 For 1 <k <n, 


F=¥ g@)-R, 6g! 1)e"@r. 


n>k 
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Figure 11.9 
A Dyck path factored into smaller paths. 


Theorem 11.3.21 is a g-analogue of the general Lagrange inversion formula [4, 
p.629] 


o0 x" n—1 
fee) = f+ [A ereoren)] 1.60) 


n=1 n! (x)" x=0 


where @ and f are analytic in a neighborhood of 0, with @(0) 4 0. To see why, 
assume without loss of generality that f(0) = 1, and set f(x) = Y?_ocxx* and @ = 
ain = LiLo ee in (11.60) to get 


H(—x) 
co oo yi q’-} co ne co 
YY cyx* = y mt a)"G I y keyx* (y emx")” | |x=0 
k=1 nee fa k=l m=0 
co xh oo 
= 3 (4) Yi kex(n— 1)!( y Cmx")" | snk 
n=1"" k=1 m=0 
=S v'H(-x9)" Yc : ej,e e 
= Rh Figs Jn 
n=1 k=1 Jitjot..+jn=n-k 


n n . k n 
= ey) ce y betas iis) (11.61) 


716 Handbook of Enumerative Combinatorics 


The equivalence of (11.61) to the g = 1 case of Theorem 11.3.21 (with c;, replaced 
by c;/ex) will follow if we can show that for any fixed a@ Fn —k, 
Fy alk eta)mta)nta),..) 
nel (k) n\k—€(a),n1(),n2(a),...)” 
B(x)—k=a 


(11.62) 


where B —k is the partition obtained by removing one part of size k from B. See [33] 
for an inductive proof of (11.62). 

In [40] Haiman includes a discussion of the connection of Theorem 11.3.20 to 
q-Lagrange inversion formulas of Andrews, Garsia, and Gessel [3, 23, 29]. Fur- 
ther background on these formulas is contained in [61]. Garsia and Haiman used 
q-Lagrange inversion to obtain the interesting identity 


Vena = Ya Megay (11.63) 


TELiin 


for the t = 1 case of the Frobenius series. 
Garsia and Haiman were also able to obtain the t = 1/q case of -¥(DH,;4q,t), 
which can be expressed as follows. 


Theorem 11.3.23 The following holds: 


n 1 
gq) F(DHy:4,1/4) = waren #Y), (11.64) 


where Y = {1,q,...,q"}. Equivalently, by the Cauchy identity (11.34), 


n 1 
(g).F (Dasa, 1/4).82) = prea. : ,q'). (11.65) 


Note that by Theorem 11.2.20, the special case A = 1” of (11.65) reduces to (11.45), 
the formula for MacMahon’s maj-statistic g-Catalan. 


Problem 11.3.24 Find a q,t-version of the Lagrange inversion formula that will 
yield an identity for ¥(DHy;q,t), and that reduces to Theorem 11.3.21 when t = 1 
and incorporates (11.64). 


11.4 Parking functions and the Hilbert series 
11.4.1 Extension of the dinv statistic 


Another beautiful corollary of Haiman’s formula for ¥ (DH); q,t) is that the dimen- 
sion of DH, equals (n+ 1)"~!, which is the number of parking functions on n cars. 
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See the chapter in this volume on parking functions for more information on these 
important combinatorial objects. In this chapter we will view parking functions geo- 
metrically, as a Dyck path z together with a placement of the numbers, or “cars,” 1 
through n in the squares just to the right of N steps of 7, with strict decrease down 
columns. See Figure 11.10. 


NY | Ni natn 


So A oe. 7 3 88 


Figure 11.10 
A parking function P. 


We now describe an extension of the dinv statistic to parking functions. Let Y, 
denote the set of all parking functions on n cars. Given P € #,, with associated Dyck 
path a = 2(P), if car iis in row j we say occupant(j) =i. Let dinv(P) be the number 
of pairs (i,j), 1 <i< j <nsuch that 


dinv(P) = |{(i,j): 1<i<j<n, a;=aj, and occupant(i) < occupant(/)}| 
+|]{G jf): 1<i<j<n, aj=aj;+1, and occupant(i) > occupant(/)}. 


Thus dinv(P) is the number of pairs of rows of P of the same length, with the row 
above containing the larger car, or which differ by one in length, with the longer row 
below the shorter, and the longer row containing the larger car. For example, for the 
parking function in Figure 11.10, the inversion pairs (i,j) are (1,7), (2,7), (2,8), 
(3,4), (4,8) and (5,6), so dinv(P) = 6. 

We define area(P) = area(z), and also define the reading word of P, denoted 
read(P), to be the permutation obtained by reading the cars along diagonals in a 
southwest direction, starting with the diagonal farthest from the line y = x, then work- 
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ing inwards. For example, the parking function in Figure 11.10 has area 9 reading 
word 64781532. 


Remark 11.4.1 Note that read(P) = n---21 if and only if dinv(P) = dinv(z). We 
call this parking function the Maxdinv parking function for 7, which we denote by 
Maxdinv(z). 


Recall that Remark 11.2.29 implies 
H(DEn 34,1) = (F (DEns4,t), hin) . (11.66) 


In [38] N. Loehr and the author advance the following conjectured combinatorial 
formula for the Hilbert series, which is still open. 


Conjecture 11.4.2 


HD ats Y GPP), (11.67) 
PEAY, 


Conjecture 11.4.2 has been verified in Maple for n < 11. The truth of the conjecture 
when q = | follows easily from (11.63) and (11.66). Later in this section we will 
show (Corollary 11.4.9) that dinv has the same distribution as area over Y,, which 
implies the conjecture is also true when ¢ = 1. 


11.4.2 Anexplicit formula 


Given Tt € S,, with descents at places ij < iz <... < ix, we call the first 7) letters of 
Tt the first run of T, the next i2 — i, letters of T the second run of T, ..., and the last 
n—ix letters of t the (k + 1)th run of 7. For example, the runs of 58246137 are 58, 
246 and 137. It will prove convenient to call element 0 the (k+2)th run of 7. Let 
cars(T) denote the set of parking functions whose cars in rows of length 0 consist of 
the elements of the (k + 1)th run of 7 (in any order), whose cars in rows of length 1 
consist of the elements of the kth run of Tt (in any order), ..., and whose cars in rows 
of length k consist of the elements of the first run of t (in any order). For example, 
the elements of cars(31254) are listed in Figure 11.11. 

Let Tt be as above, and let i be a given integer satisfying | <i <n. If 7 is in the jth 
run of t, we define w;(t) to be the number of elements in the jth run that are larger 
than 7;, plus the number of elements in the (j + 1)th run that are smaller than 7;. For 
example, if tT = 385924617, then the values of w1,w2,...,wo are 1,1,3,3,3,2,1,2,1. 


Theorem 11.4.3 Given Tt € Sy, 


Y giim(P) garea(P) — pmai() [wi(T)]q- (11.68) 


Pecars(T) i=l 


Proof. We will build up elements of cars(t) by starting at the end of t, where 
elements go in rows of length 0, and add elements right to left. We define a partial 
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3 9 
2 3 
1 
5 5 
4 4 
3 1 
1 3 
2 2 
5 5 
4 4 


Figure 11.11 
The elements of cars(31254). 


parking function to be a Dyck path z € L;_,, for some m, together with a placement 
of m distinct positive integers (not necessarily the integers 1 through m) to the right 
of the N steps of 7, with strict decrease down columns. Say tT = 385924617 and we 
have just added car 9 to obtain a partial parking function A with cars 1 and 7 in rows 
of length 0, cars 2, 4 and 6 in rows of length 1, and car 9 in a row of length 2, as 
in the upper-left grid of Figure 11.12. The rows with *’s to the right are rows above 
which we can insert a row of length 2 with car 5 in it and still have a partial parking 
function. Note the number of starred rows equals w3(t), and that in general w;(t) 
can be defined as the number of ways to insert a row containing car 7; into a partial 
parking function containing cars T+1,...,T, in rows of the appropriate length, and 
still obtain a partial parking function. 

Consider what happens to dinv as we insert the row with car 5 above a starred 
row to form A’. Pairs of rows that form inversions in A will also form inversions in A’. 
Furthermore the rows of length 0 in A, or of length | with a car larger than 5, cannot 
form inversions with car 5 no matter where it is inserted. However, a starred row will 
form an inversion with car 5 if and only if car 5 is in a row below it. It follows that 
if we weight insertions by g“"’, inserting car 5 at the various places will generate a 
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ei 
7 
5 
4 k 
A 
9 x 
6 9 
6 
2 a 
1 ps 
ie i 
dinv = dinv (A) +0 
ey 7 
4 4 
5 9 
9 6 
6 5 
2 2 


dinv = dinv (A) + 1 dinv = dinv (A) + 2 


Figure 11.12 
Partial parking functions occurring in the proof of Theorem 11.4.3. 


factor of [w;(T)] times the weight of A, as in Figure 11.12. Finally note that for any 
path z corresponding to an element of cars(t), maj(t) = area(z). a 


By summing Theorem 11.4.3 over all tT € S, we get the following. 


Corollary 11.4.4 The identity 


y gan) pea) = »: pnai(t) [wi(T)]q- 


PEPy TESn i=1 


holds. 
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Problem 11.4.5 Prove 


Y qflinv(P) garealP) (11.69) 
PEP» 
is symmetric in q,t. 


Remark 11.4.6 Beyond merely proving the symmetry of (11.69), one could hope 
to find a bijective proof. It is interesting to note that by (11.68) the symmetry in q,t 
when one variable equals 0 reduces to the fact that both inv and maj are Mahonian. 
Hence any bijective proof of symmetry may have to involve generalizing Foata’s 
bijective transformation of maj into inv. 


11.4.3 The statistic area’ 


By a diagonal labeling of a Dyck path z € L;*,, we mean a placement of the numbers 
1 through n in the squares on the main diagonal y =x in such a way that for every 
consecutive EN pair of steps of 7, the number in the same column as the E step is 
smaller than the number in the same row as the N step. Let .</, denote the set of pairs 
(A, 2) where A is a diagonal labeling of 7 € Lj, Given such a pair (A,7), we let 
area'(A,7) denote the number of area squares x of a for which the number on the 
diagonal in the same column as x is smaller than the number in the same row as x. 
Also define bounce(A, a) = bounce(z). 

The following result appears in [38]. 


Theorem 11.4.7 There is a bijection between P;, and &, that sends (dinv, area) to 
(area' , bounce). 


Proof. Given P € #, with associated path 7, we begin to construct a pair (A, €(z)) 
by first letting ¢ (77) be the same path formed by the € map from the proof of Theorem 
11.3.13. The length a of the first bounce of ¢ is the number of rows of 7 of length 
0, etc.. Next place the cars that occur in P in the rows of length 0 in the lowest a 
diagonal squares of ¢, in such a way that the order in which they occur, reading top 
to bottom, in P is the same as the order in which they occur, reading top to bottom, 
in ¢. Then place the cars that occur in P in the rows of length | in the next a 
diagonal squares of ¢, in such a way that the order in which they occur, reading top 
to bottom, in P is the same as the order in which they occur, reading top to bottom, in 
¢. Continue in this way until all the diagonal squares are filled, resulting in the pair 
(A,¢). See Figure 11.13 for an example. 

The properties of the € map immediately imply area(z) = bounce(A, ¢) and that 
(A,¢) € &,. The reader will have no trouble showing that the equation dinv(P) = 
area'(A,C) is also implicit in the proof of Theorem 11.3.13. | 


Remark 11.4.8 Drew Armstrong [7] has found an interpretation for the area’ statis- 
tic, as well as the bounce and dinv statistics, in terms of hyperplane arrangements. 
See also [9]. 
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( dinv, area ) = (5, 13) (area , bounce ) = (5, 13 ) 


Figure 11.13 
The map in the proof of Theorem 11.4.7. Squares contributing to area’ are marked 
with x’s. 


11.4.4 The pmaj statistic 


We now define a statistic on parking functions called pmaj, due to Loehr and Remmel 
[52, 50], which generalizes the bounce statistic. Given P € Y,, we define the pmaj- 
parking order, denoted B(P), by the following procedure. Let C; = C;(P) denote 
the set of cars in column i of P, and let B, be the largest car in C}. We begin by 
parking car B; in spot 1. Next we perform the “dragnet” operation, which takes all 
the cars in C, \ {B; } and combines them with C to form C). Let By be the largest car 
in C; that is smaller than ;. If there is no such car, let By be the largest car in C}. 
Park car B2 in spot 2 and then iterate this procedure. Assuming we have just parked 
car B;-1 in spot i— 1, 3 <i <n, we letC; =C/_, \ {Bj-1} and let B; be the largest 
car in Cc! that is smaller than §;_,, if any, while otherwise f; is the largest car in C! : 
For the example in Figure 11.14, we have Cy = {5}, Cy) = {1,7}. Cs = {}, etc. and 
Ci = {1,7}, Cy = {7}, Cy = {2,4,6}, C5 = {2,3,4}, ete., with B = 51764328. 

Now let rev(B(P)) = (Bn, Bn—1,---, Bi) and define pmaj(P) = maj(rev(B(P))). 
For example, for the parking function of Figure 11.14 we have rev(B) = 82346715 
and pmaj = 1+6 =7. Given z € L;',,, it is easy to see that if P is the parking function 
for obtained by placing car i in row i for 1 <i <n, then pmaj(P) = bounce(z). We 
call this parking function the primary pmaj parking function for 7. 

We now describe a bijection T from Y, to Y, from [52] that sends (area, pmaj) 
— (dinv,area). The crucial observation behind it is this. Fix y € S, and consider 
the set of parking functions that satisfy rev(B(P)) = y. We can build up this set 
recursively by first forming a partial parking function consisting of car y, in column 
1. If %-1 < YF, then we can form a partial parking function consisting of two cars 
whose pmaj parking order is Y)%,— 1 in two ways. We can either have both cars y, 
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B= 5 176 43 2 8 


Figure 11.14 
A parking function P with pmaj parking order B = 51764328. 


and Y¥%,—; in column 1, or car y, in column | and car y%,—; in column 2. If Wr-1 > Yr, 
then we must have car y, in column | and car y,_; in column 2. In the case where 
Yr—1 < Yo, there were two choices for columns to insert car y,_1 into, corresponding 
to the fact that w,_1(Y) = 2. When %,_1 > Ym, there was only one choice for the 
column to insert %—1 into, and correspondingly wy_1(Y) = 1. 

More generally, say we have a partial parking function consisting of cars in the 
set {%m,---, +1} whose pmaj parking order is y--- 742741. It is easy to see that 
the number of ways to insert car 7; into this so the new partial parking function has 
pmaj parking order 7, --- ¥;417; is exactly w;(). Furthermore, as you insert car ; into 
columns n —i+1,n—i,...,1.—i—w;(y) +2 the area of the partial parking function 
increases by | each time. It follows that 


TT 


y ge ) zpmaj(P =D pmai(y Tbe (Y)]. 


PEP, YESn f=1 


Moreover, we can identify the values of (area, pmaj) for individual parking functions 
by considering permutations y € S,, and corresponding n-tuples (u1,...,U¥n) with 0 < 
uj < w;(y) for 1 <i <n. (Note u, always equals 0). Then maj(y) = pmaj(P), and 
uy +...+U, = area(P). (For those familiar with the description of parking functions 
in terms of preference functions, we have f(B,+1-;) =n +1—i-—u; for 1<i<n.) 
Now given such a pair y € S,, and corresponding n-tuple (u;,...,u,), from the 
proof of Theorem 11.4.3 we can build up a parking function Q recursively by insert- 
ing Cars Yj, Yn—1,--. One at a time, where for each j the insertion of car y; adds uj; 
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to dinv(Q). Thus we end up with a bijection [: P+» Q with (area(P),pmaj(P)) = 
(dinv(Q),area(Q)). The top of Figure 11.15 gives the various partial parking func- 
tions in the construction of P, and after those are the various partial parking functions 
in the construction of Q, for y = 563412 and (u1,...,u6) = (2,0,1,0,1,0). 


2 4 
e-o— et 


1 2 : 3 


Ww 


Figure 11.15 
The recursive construction of the P and Q parking functions in the I’ correspondence 
for y = 563412 and u = (2,0,1,0,1,0). 


Corollary 11.4.9 The marginal distributions of pmaj, area, and dinv over Py are 
all the same, i.é., 


y gral?) is y gr) = y ga. 


PEP, PEP, PEP, 


Exercise 11.4.10 Notice that in Figure 11.15, the final correspondence is between 
parking functions that equal the primary pmaj and Maxdinv parking functions for 
their respective paths. Show that this is true in general, i.e., that when P equals the 
primary pmaj parking function for 7% then the bijection T reduces to the inverse of 
the bijection € from the proof of Theorem 11.3.13. 
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11.4.5 The cyclic-shift operation 


Given S C {1,...,n}, let A,,5 denote the set of parking functions for which C; (P) = 
S. If x € {1,2,...,n}, define 


ies i ifx<n 

1 ifx =n. 
For any set SC {1,2,...,n}, let CYC,(S) = {CYC,(x) : x € S}. Assume S = {51 < 
82 <-+++ < sg} with sy <n. Given P € ys, define the cyclic-shift of P, denoted 
CYC, (P), to be the parking function obtained by replacing C;, the cars in column i 
of P, with CYC,(C;), for each 1 <i <n. Note that the column of P containing car 
n will have to be sorted, with car 1 moved to the bottom of the column. The map 
CYC, (P) is undefined if car n is in column 1. See the top portion of Figure 11.16 for 
an example. 


4 5 1 
5 2 3 
| | | | 2 | 
3 4 5 
2 3 4 
P CYC(P) cyc? (P) 
1 
4 
3| | 
2 
R(P) 
Figure 11.16 
The map R(P). 


Proposition 11.4.11 [50] Suppose P € Py with S = {1 < 52 <--> < seh, se <n. 
Then 


pmaj(P) = pmaj(CYC(P)) +1. (11.70) 
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Proof. Imagine adding a second coordinate to each car, with car i initially repre- 
sented by (i,i). If we list the second coordinates of each car as they occur in the pmaj 
parking order for P, by definition we get the sequence f; (P)B.(P)---B,(P). We now 
perform the cyclic-shift operation to obtain CYC(P), but when doing so we operate 
only on the first coordinates of each car, leaving the second coordinates unchanged. 
The reader will have no trouble verifying that if we now list the second coordinates 
of each car as they occur in the pmaj parking order for CYC(P), we again get the 
sequence §(P)B2(P)---Bn(P). It follows that the pmaj parking order of CYC(P) 
can be obtained by starting with the pmaj parking order B of P and performing the 
cyclic-shift operation on each element of B individually. See Figure 11.17. 


(3,3) (2) dU.) (55) (44) —~—(43) ©@.2) @1) 5) G64) 


Figure 11.17 
The cyclic-shift operation and the pmaj parking order. 


Say 7 occurs in the permutation rev(f$(P)) in spot j. Note that we must have 
j <x, or otherwise car n would be in column | of P. Clearly when we perform the 
cyclic-shift operation on the individual elements of the permutation rev(B(P)) the 
descent set will remain the same, except that the descent at j is now replaced by a 
descent at j — 1 if 7 > 1, or is removed if j = 1. In any case the value of the major 
index of rev(B(P)) is decreased by 1. | 


Using Proposition 11.4.11, Loehr derives the following recurrence. 


Theorem 11.4.12 Letn> 1 and S = {5 <-++ < sg} C {1,...,n}. Set 


Pysta.t)= Yi giteolPhermal), 
Pe Pns 
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Then 


_ ,k-1,n-s, 
Rslahag- re yy Phuc nec aimieaay i> (11.71) 


with the initial conditions P,,9(q,t) =0 for all n and P, 51}(q,t) = 1. 


Proof. For PE Py s,letQ=CYC, “*(P). Then 
pmaj(Q) + — sx = pmaj(P) 
area(Q) = area(P). 


Since car n is in the first column of Q, in the pmaj parking order for Q car n is in 
spot 1. By definition, the dragnet operation will then combine the remaining cars in 
column | of Q with the cars in column 2 of Q. Now car n being in spot | translates 
into car n being at the end of rev(B(Q)), which means n — 1 is not in the descent set 
of rev(B(Q)). Thus if we define R(P) to be the element of Y,_; obtained by parking 
car n in spot 1, performing the dragnet, then truncating column | and spot 1 as in 
Figure 11.16, we have 


pmaj(R(P)) = pmaj(P) — (n— sx). 


Furthermore, performing the dragnet leaves the number of area cells in columns 
2,3,...,n of Q unchanged but eliminates the k — | area cells in column 1| of Q. Thus 


area(R(P)) = area(P) —k+1 
and the recursion now follows easily. | 


Loehr also derives the following compact formula for P,,5 when t = 1/q. 


Theorem 11.4.13 Forn > 0Oand S = {51 <-++< sx} C {1,...,n}, 


q) Pys(1/q.q) = 4" [nl 1 YY gh. (11.72) 
xeES 


Proof. Our proof is, for the most part, taken from [50]. If k =n, 


Pras(1/4.4) = Patt, (L/qa) =¢ ©), 


while the right-hand side of (11.72) equals {n]~!{n] = 1. Thus (11.72) holds for k =n. 
It also holds trivially for n = 0 and n = 1. So assume n > 1 and 0 <k <n. From 
(11.71), 


Psijggagi ey, Pere? sur\{u}) 1/4 (11.73) 


— pnt l—sp—k 
=q . P-ucyel*(sur\{s}) 4/49: 
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The summand when j =n —k equals 


n—1 
Figg Ga) = gi). 


For 0 < j <n—k, by induction the summand equals 


gh GHD ("') in — "ime ae y ane (11.74) 
xeCY¥C, “(SUT \{s}) 
since j+k—1=|CYC; “(SUT \ {s;})|. Plugging (11.74) into (11.73) and reversing 
summation we now have 


n—k-1 n—1 
gq 2ntsetkg(2) P, s(1/q,q) =1]4+ y gq’ iin — Wer y "ie (11.75) 
j=0 x=1 


x Ex (TC {,...,n}\S,|T] = f,CYC# (x) € SUT \ {s}). 
Lr 


To compute the inner sum over T above, we consider two cases. 


1. x =n— (sx —5;) for some i < k. Since x <n, this implies i < k, and since 
CYC; " (x) =5;, we have CYC;!-"(x) € SUT \ {s,}. Thus the inner sum above 
equals the number of j-element subsets of {1,...,}\S, or (3). 


2. x An—(s,—5;) for all i < k. By Exercise 11.4.14 below, the inner sum over T 
in (11.75) equals (">"). 


Applying the above analysis to (11.75) we now have 


n—k-1 
gq 2 ttkg®) P, s(1/q,q) =1+4+ y gk [n -_ irs 
j=0 


a emery aan 
Xx satisfies (1) J j-l 


n—k—1 = ear n—-k—1 i 
ai y gq’ ~In— 1 oot y ( 2 a’ a 


j=0 x satisfies (2) J 


Now x satisfies (1) if and only ifn — 1—x = 5, —s;— 1 for some i < k, and so 


~2n-+5,+k (3) ae n—-k-j n—k-j n—k—-1 
q q2)Pys(1/q,q)=1+ Yo qh * J [n- I] jo 
j=0 


a ee In 2 ‘ieee (" —k- ) yee 


j=0 J 


n—k-1 ae 
= y (" k 'Y (an yore 
m=0(m=j—1) 


m 
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k-1 n—k—1 ray ee 
See » (ain— ay 1( : ' 
i=1 


j=0 J 


k-1 
= (1+ git Fg (1 gl — 1 
i=1 


k-1 
= ny 4 y gt“), 
i=1 


Thus 


GP, s(1/q.9) =a np! ger. 
xeS 


Exercise 11.4.14 Show that if x An — (s,—s;) for alli <k, the inner sum over T in 
(11.75) equals ("|*;"). 


As a corollary of his formula for *(DH,;q,t), Haiman proves a conjecture he 
attributes in [40] to Stanley, namely that 


gq?) C(DHy31/q,q) = (n+ Ay. (11.76) 


Theorem 11.4.13 and (11.76) together imply Conjecture 11.4.2 is true when t = q 
and gq = 1/q. To see why, first observe that 


Pasi {nti} (ot) = »y great?) pmal(), 
PEP, 
Hence by Theorem 11.4.13, 
g®) Yq teal) gomai(P) — q) Piss ngs) (1/4,4) 
PEP, 
_ n+l 
agg’? Pras enasy (1/49) 
= gq "q'[n+ it = [n+ rae 


The main impediment to proving Conjecture 11.4.2 seems to be the lack of a 
recursive decomposition of the Hilbert series along the lines of (11.43). 


11.4.6 Tesler matrices 


For ann x n upper triangular matrix, we define the jth hook sum, where | < j <n, 
to be the sum of all the entries in the jth row of the matrix, minus the sum of all 
the entries in the jth column strictly above the diagonal. A Tesler matrix of order 
nis ann Xn upper-triangular matrix of nonnegative integers, such that all the hook 
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sums equal 1. Let Tes(n) denote the set of Tesler matrices of order n. For example, 
the elements of Tes(3) are 


1 0 0] fl 0 0] fo 1 0] fo 0 
Oi OG OPO: Dl )0..2. Ol. 0: I 1), (11.77) 
O00) 0. O21. 0.0: AP Oe 

0 1 0] fo 0 1] fo o 

0 0 2],/0 1 o},]0 0 

00 3) |O oO 2] lo Oo 3 


Let [k]gx = (t* — q*)/(t —q) denote the q,t-analog of the integer k, and recall that 
M = (1—q)(1 —1). To each Tesler matrix C we associate the weight 


wt(C) = (apo Il Iciilats 


cij>0 


where pos(C) is the number of positive entries in C. For example, the weight of 


0 1 0 
0 1 1 
0 0 2 


is (¢+q)(—M) = —(t+q)(1—g) (1-2). 

Using the theory of Macdonald polynomials and (11.40), Haglund [35] proved 
the following result, which gives a formula for #(DH,,;q,t) in terms of Tesler ma- 
trices. 


Theorem 11.4.15 /35] The following equality holds: 


HC(DHng,t)= )Y wt(C). (11.78) 
Ce Tes(n) 


Example 11.4.16 When n = 3, the terms in (11.77), with weights, give 


He (DH33q,t) =1+ (t+q)+(t+4)-(l-q) -)(t+4)+ 
G4 Het) +e tote tig te). 


Note that Formula (11.78) is clearly a polynomial, and clearly symmetric in q,t. 
It gives a possible way of attacking Conjecture 11.4.2 without the use of symmetric 
function theory, since in principle one could figure out how to cancel the negative 
terms in the right-hand side of (11.78), leaving a positive expression as in Conjecture 
11.4.2. P. Levande [47, 48] has shown how to do this cancellation when t = | and 
when ¢t = 0. 


Catalan Paths and q,t-enumeration 731 


11.5 The g,t-Schréder polynomial 
11.5.1 The Schroder bounce and area statistics 


In this section we develop the theory of the g,t-Schréder polynomial, which gives a 
combinatorial interpretation, in terms of statistics on Schroder lattice paths, for the 
coefficient of a Schur hook shape in ¥ (DH,,;q,t). A Schréder path is a lattice path 
from (0,0) to (n,n) consisting of N(0,1), Z(1,0) and diagonal D(1,1) steps that 
never goes below the line y = x. We let if ng denote the set of Schréder lattice paths 
consisting of n—d N steps, n—d E steps, and d D steps. We refer to a triangle whose 
vertex set is of the form {(i,j),(i+1,j),(i+1,j+1)} for some (i,j) as a lower 
triangle, and define the area of a Schréder path 7 to be the number of lower triangles 
below z and above the line y = x. Note that if z has no D steps, then the Schroder 
definition of area agrees with the definition of the area of a Catalan path. We let a;(z) 
denote the length of the ith row, i.e., the number of lower triangles between the path 
and the diagonal in the ith row from the bottom of , so area(z) = 2, aj(2). 
Given 7 € cA let 0 (7) be the word of 0’s, 1’s and 2’s obtained in the following 
way. Initialize © to be the empty string, then start at (0,0) and travel along z to 
(n,n), adding a 0, 1, or 2 to the end of o(z) when we encounter a N, D, or E step, 
respectively, of 7. (If 2 is a Dyck path, then this definition of o (7) is the same as the 
previous definition from Section 11.2, except that we end up with a word of 0’s and 
2’s instead of 0’s and 1’s. Since all our applications involving o(7) depend only on 
the relative order of the elements of o(7), this change is only a superficial one. We 
define the statistic bounce(7) by means of the following algorithm. 


Algorithm 11.5.1 


1. First remove all D steps from 1, and collapse to obtain a Catalan path (2). 
More precisely, let 1 (2) be the Catalan path for which o(T'(2)) equals o(2) 
with all 1’s removed. Recall the ith peak of T(2) is the lattice point where 
the bounce path for T(x) switches direction from N to E for the ith time. The 
lattice point at the beginning of the corresponding E step of 7 is called the ith 
peak of %. 


2. For each D step x of 1, let nump(x) be the number of peaks of 1 below x. Then 
define 


bounce(m) = bounce(T(m)) +) nump(x), 


where the sum is over all D steps of 7. For example, if ™ is the Schroder 
path on the left in Figure 11.18, with T(a) on the right, then bounce(1) = 
(3+1)+(0+1+1+2) =8. Note that if m has no D steps, this definition of 
bounce(1) agrees with the previous definition from Section 11.3. 


732 Handbook of Enumerative Combinatorics 


Figure 11.18 
On the left, a Schréder path 7 with the peaks marked by large dots. On the right is 
I'(z) and its bounce path and peaks. 


We call the vector whose ith coordinate is the length of the ith bounce step of 
T(z) the bounce vector of z. Say I'(z) has b bounce steps, and call the set of rows 
of 2 between peaks i and i+ 1 section i of 2 for 1 <i < b. In addition we call section 
0 the set of rows below peak 1, and section b the set of rows above peak b. If 7 has 
B; D steps in section i, 0 <i < b, we refer to (Bo, Bi,..., By) as the shift vector of 7. 

For example, the path on the left in Figure 11.18 has bounce vector (2,2, 1) and 
shift vector (1,2,1,0). We refer to the portion of o(7) corresponding to the ith sec- 
tion of 7 as the ith section of o(7). 

Given n,d € N, we define the g,t-Schréder polynomial S,, 7(q,t) as follows. 


d (q,t = 2 gr ) goounce(z) 
Sn, d 


meELy nd 


These polynomials were introduced by Egge, Haglund, Killpatrick and Kremer [18]. 
They conjectured the following result, which was subsequently proved by Haglund 
using plethystic results involving Macdonald polynomials [34]. 


Theorem 11.5.2 For all0 <d <n, 
Sna(qst) = (¥ (DHn3q,t),en—aha) : 
Since 


Sn,0(4,t) = Fn(q,t) = Cr(q,t), 
the d = 0 case of Theorem 11.5.2 reduces to Theorem 11.3.3. 
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Let Le ha denote the set of paths 7 which are in aie d and also have no D step 


above the highest N step, i.e., no 1’s in o(7) after the rightmost 0. Define 


area( ) pbounce 1 
Sud (q,t) ee q ( x, 
meLt 


nnd 


Then the following holds. 


Theorem 11.5.3 Theorem 11.5.2 is equivalent to the statement that for allO <d< 
n—l, 


Sulat)= (F (DHn4.1),Say1m-t-1) 


Proof. Given m € L* nar We can map 7 to a path @(7) € ee a+1 by replacing the 
highest N step and the following E step of z by aD step. By Exercise 11.5.4 below, 


this map leaves area and bounce unchanged. Conversely, if @ € Li wal has a D step 


above the highest N step, we can map it to a path z € L* n.nd—| i an area and bounce 
preserving fashion by changing the highest D step to a NE pair. It follows that for 
l<d<n, 


Sna(qt) = y gree m) -bounce(x m7) 4 y gene pone) 


nel, sn,d—1 


ad Sna(qst) + S,,4-1(9,t). 


Since Sno(q,t) = Sno(q,t), €n—aha = Saqijn-a-1 + Sq na for 0 <d <n—1, and 
Snal(g,t) =1= S541 (q,t), the result follows by a simple inductive argument. | 


Exercise 11.5.4 Given 1 and a@(1) in the proof of Theorem 11.5.3, show that 


area(%) = area(a(7)) 
bounce(m) = bounce(a(7)). 


Define g,t-analogues of the big Schroder numbers r,, and little Schroder num- 
bers 7,, as follows. 


n 
t)= 2X Sna(aet) 
Fn (q,t) =F Snal qt) 


The numbers r,(1,1) count the total number of Schréder paths from (0,0) to (7,7). 
The 7,(1,1) are known to count many different objects [58, p.178], including the 
number of Schréder paths from (0,0) to (1,7) that have no D steps on the line y = x. 
From our comments above we have the simple identity r,,(q,t) = 27,(q,t), and using 
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Haiman’s formula for Y (DH,;9,t) we get the polynomial identities 


a 
Ty Wren (w +4 t MIL, By 


y w*Sna(qst) = ‘Ss 


d=0 urn Wu 

HEN 5 i w+qt" )MIL,B 

¥ w4s,a(gt)= y+ Teneo Fat MMB yn (11.79) 
d=0 en Wu 


An interesting special case of (11.79) is 


/ - 
Tu [Maat -¢4 a )MBu 


Wu 


Fna(q,t) = y 


brn 


11.5.2 Recurrences and explicit formulae 


We begin with a useful lemma about area and Schréder paths. 


Lemma 11.5.5 (The “boundary lemma”) Given a,b,c € N, let boundary(a,b,c) 
be the path whose o word is 2°1°0%. Then 


yee) = |" | 
Tt q 


a,b,c 


where the sum is over all paths 1 from (0,0) to (c-++b,a+b) consisting of a N steps, 
b D steps and c E steps, and area' (1) is the number of lower triangles between 1 
and boundary(a,b,c). 


Proof. Given 1 as above, we claim the number of coinversions of o(7) equals 
area’ (7). To see why, start with z as in Figure 11.19, and note that when consecutive 
ND, DE, or NE steps are interchanged, area’ decreases by 1. Thus area’(z) equals 
the number of such interchanges needed to transform z into boundary(a,b,c), or 
equivalently to transform o(7) into 2°170*. But this is just coinv(o(z)). Thus 


y que) = y groinv(o) 
1 


GEM (a.b,c) 


a y gg) 


OEM (a,b,c) 


a+b+c 
a,b,c 4 


by (11.2). | 


Given n,d,k € N with 1 <k <n, let Ly alk) denote the set of paths in L*_, that 


‘n,n,d 


have k total D plus N steps below the lowest E step. We view L7,,_,(k) as containing 


nnn 


Catalan Paths and q,t-enumeration 735 


TU 


\ 


boundary (4,3,5) 


Figure 11.19 
The region between a path z and the corresponding boundary path. For this region 
area’ = 27. 


the path with n D steps if k =n and L,*,, ,, (k) as being the empty set if k < n. Define 


nnn 


Snak(qst) = y Gar Poe (11.80) 


oi 
TEL nd (k) 


with Spnk(q,t) = 4 (k =n). There is a recursive structure underlying the S,,ax(q,t) 
which extends that underlying the Fik(,t)- The following result is derived in [34], 
and is similar to recurrence relations occurring in [18]. For any two integers n,k we 
use the notation 


On,k = x(n = k). 
Theorem 11.5.6 Letn,k,d © N with 1 <k <n. Then 
Snink(st) = 9n,k; (11.81) 


and forO <d <n, 


Sanday" 


P 


min(k,n—d) t 
) J 


n—k : 
P +j-1 
| gQ2) (i : | Sn—kad-p-ki(Gst), (11.82) 
p=max(1,k—d q j=0 q 
with the initial conditions 


Sook = 9k,0;  Sna,o = 6n,08a.0- 


Proof. If d =n then (11.81) follows directly from the definition. If d < n then 
m has at least one peak. Say 2 has p N steps and k — p D steps in section 0. First 
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assume p < n—d, i.e., II has at least two peaks. We now describe an operation we 
call truncation, which takes 7 € ea q(k) and maps it toa a’ € Lt ken-kd—k-p With 
one less peak. Given such a 7, to create 7’ start with o(a) and remove the first 
k letters (section 0). Also remove all the 2’s in section 1. The result is o(2’). For 
example, for the path on the left in Figure 11.18, o(7) = 10020201120212, k = 3 
and o(z’) = 001120212. 

We will use Figure 11.20 as a visual aid in the remainder of our argument. Let j 
be the total number of diagonal and north steps of z in section | of z. By construction 
the bounce path for I’'(z’) will be identical to the bounce path for I'(II) except the 
first bounce of T(z) is truncated. This bounce step hits the diagonal at (p,p), and 
so the contribution to bounce(z’) from the bounce path will be n — d — p less than 
to bounce(z). Furthermore, for each D step of a above peak 1 of II, the number 
of peaks of 2’ below it will be one less than the number of peaks of 7 below it. It 
follows that 


bounce(z) = bounce(z’) +n—d — p+d-—(k—p) 


= bounce(z’) +n—k. 
Since the area below the triangle of side p from Figure 11.20 is ae 


P 


area(m) = area(z’) + (! 


) + area0 + areal, 

where area0 is the area of section 0 of 7, and area! is the portion of the area of section 
1 of z not included in area(z’). When we sum q**°°(”) over all  € L* _,(k) that get 
mapped to 2’ under truncation, we generate a factor of 


ul 


by the c = 0 case of the boundary lemma. 

From the proof of the boundary lemma, areal equals the number of coinversions 
of the first section of o(7) involving pairs of 0’s and 2’s or pairs of 1’s and 2’s. We 
need to consider the sum of g to the number of such coinversions, summed over all 
7 that map to 2’, or equivalently, summed over all ways to interleave the p 2’s into 
the fixed sequence of j 0’s and 1’s in section 0 of z’. Taking into account the fact 
that such an interleaving must begin with a 2 but is otherwise unrestricted, we get a 


factor of 
? -1 *) 
J bs 


since each 2 will form a coinversion with each 0 and each 1 occurring before it. It is 
now clear how the various terms in (11.82) arise. 

Finally, we consider the case when there is only one peak, so p = n—d. Since 
there are d— (k— p) =d—k+n—d=n—k D steps above peak | of z, we have 
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Region 1 


Region 0 


% 


Figure 11.20 
A path z decomposed into various regions under truncation. 


bounce(z) =n-—k. Taking area into account, by the above analysis we get 


k =d@-14-n=k 
Sndk(q,t) =t"” kgs | | " ete a | (11.83) 
n—d\, n—k 


which agrees with the p = n—d term on the right-hand side of (11.82) since 
Sn—kn—k,j(Qot) = 9) nx from the initial conditions. | 


The following explicit formula for S,4(q,t) was obtained in [18]. 


Theorem 11.5.7 For all0<d <n, 


Snd (q,t) -¥ 


b=1 a) 4+...+Q,=n—d, a;>0 | Bo 
Bo+Bi +.-.+Bp=d, Bj20 


Bo+ i é - - ' at) 


(11.84) 


1P1 2Bo+...+bBp+02+203+...+(b Me | 
Bi, O41, a= 1 qd 
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Proof. Consider the sum of g*@s>°unce over all 2 that have bounce vector 
(Q1,.--,@), and shift vector (Bo, Bi,...,B,). For all such 2 the value of bounce 
is given by the exponent of f in (11.84). The area below the bounce path gener- 
ates the ghar) term. When computing the portion of area above the bounce 


path, section 0 of z contributes the [ oe term. Similarly, section b contributes the 
q 


aaa term (the first step above peak b must be an E step by the definition of a 


peak, which explains why we subtract | from q,). For section i, 1 <i <b, we sum 
over all ways to interleave the 8; D steps with the a4; N steps and the a; E steps, 


subject to the constraint we start with an E step. By the boundary lemma, we get the 
ears 


term. 
Bi,Oj41,04—-1 It : 


11.5.3 The special value ¢ = 1 /q 
Theorem 11.5.8 For1<k<nand0<d<n, 


(3)-(3) __(k-1(n—ay [K] [2n-k-—d—1] [n 
q Snak(q:1/q) =4 “| pan |. (11.85) 


Sketch of proof. |The result can be obtained by induction, as in the case of the proof 
of Theorem 11.3.8. The details of this argument can be found in [39, pp. 54-56]. 


Corollary 11.5.9 ForO0<d<vn, 


n\_(d 1 2n—d 
gq) @) 8, a(q,1/q) = aren eee (11.86) 


Proof. It is easy to see combinatorially that S,+1,4,1(¢,t) =t"Sn.a(q,t). Thus 
n d n d 
gC) S,,a(4,1/4) =a" Sn41.41(4 1/4) 


n+1 


= gi y i rem (q,1/q) 


corre LM [aera] ea 
q q 


™ [In+1] n d 


_ 1 2n—d 
~ [n—d+1]|n—d,n—d,d Ps 
| 


Remark 11.5.10 Corollary 11.5.9 proves that S,4(q,t) is symmetric in g,t when 
t = 1/q. For we have 


js ia 1 2n—d 
Sna(q;1/9) =4 Oa Eade 
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and replacing q by 1/q we get 


? 


5)- a qq 2n—d ne Q es od 
Snelling =a Re a aa] PEO CE) 


since [n!1/g = [n] Ig), Now 


n d yee ee n—d & d 2n—d\ (d n 

a ey a 2 2 ae Oa a ee 
80 Sn.a(q,1/q) = Sna(1/q,q). It is of course an open problem to show S,,a(q,t) = 
Sna(t,q) bijectively, since the d = 0 case is Open Problem 11.3.10. 


11.5.4 A Schroder dinv-statistic 
Let 


n 
Cn(q,t,w) = y w'Sna(qst), 
d=0 


and for 7 € ee let a(7) equal the number of area squares in the ith column from the 
right. Also set ag(z) = —1 for all z. For example, for the path on the right in Figure 
11.8, we have (ap,a\,a5,...,ag) = (—1,0,1,1,2,3,3,1,0). The (¢,t)-Schréder can 


be expressed strictly in terms of Dyck paths as follows. 


Theorem 11.5.11 The following equality holds: 


Cn(q,t,w) ae y gre) poe) Il (1 +w/q"). (11.87) 
TELitn I<i<n 
ai>a\_, 


Proof. (sketch) Show the coefficient of w? in (11.87) satisfies the same recurrence 
as (11.82). a 


Next define the reading order of the rows of 7 to be the order in which the rows 
are listed by decreasing value of a;, where if two rows have the same aj-value, the 
row above is listed first. For example, for the path on the left in Figure 11.8, the 
reading order is 


row 6, row 8, row 5, row 4, row 3, row 7, row 2, row 1. 


Finally let b, = b; (2) be the number of inversion pairs as in Definition 11.3.12 which 
involve the kth row in the reading order and rows before it in the reading order, and 
set by) = —1. For example, for the path on the left in Figure 11.8, we have 


(bo, b1,...,bg) = (—1,0, 1, 1,2,3,3, 1,0). (11.88) 
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Note that in this example a/(¢(2)) = b;(7) for all i, and it is not hard to see that this 
is true in general. Hence we have 


n(qtw) = y giiv™zareal™) TT (1+w/q’). (11.89) 
TEL ISi<n 
. bj>bj_1 


It is also easy to see that b; > bj_, if and only if row; contains a column top, i.e., the 
N step of the path in that row is followed immediately by an E step. 

Now to get a term in (11.89) contributing to S,4(g,t) we make a selection of 
d column tops, and for the rows containing those column tops we subtract the cor- 
responding b; contribution to dinv. If we start with the path z and replace those 
column tops by D steps, we get a Schréder path, and we can define dinv of this path 
as dinv(z) minus the sum of the chosen Jj. 


11.5.5 The shuffle conjecture 


In [37] a nice conjecture for the expansion of ¥(DH,;q,t) into monomials is in- 
troduced. It is often referred to as the shuffle conjecture, since it can be phrased 


in the following simple way: decompose {1,2,...,} into increasing sequences of 
consecutive integers 1, Qo,..., 0% of lengths ||, |Q2|,... and decreasing sequences 
of consecutive integers B),...,B,; of lengths |B1|,|B2|,.... For example, if n = 8 we 
might have 

k=1,8=2, 0) = {6,7,8},Bi = {5,4,3}, Bo = {2,1}. (11.90) 


Given such a decomposition, we say a permutation o € S,, is an a, B-shuffle if for 
each i,j all the terms in Qj occur in increasing order in o, and all the terms of B; 
occur in decreasing order in o. For example, for a, B as in (11.90), the permutation 
26571483 is an a, B-shuffle. 


Conjecture 11.5.12 (The Shuffle Conjecture [37]) 


(F (DEbnsdet) Ho tag|“Mayleigieital eB) =D ala a 
EP, 
read(P) is an &, B-shuffle 
(11.91) 
If our decomposition of {1,2,...,} is into n sequences consisting of only one ele- 


ment each, then the set of a, B-shuffles is the set of all parking functions. By Remark 
11.2.29 the left-hand side of (11.91) gives #(DH,,;q,t), and so in this case the shuf- 
fle conjecture reduces to Conjecture 11.4.2. On the other hand, consider the case 
k=s=1,a,=n—d+l,...,n, Bj =n—d,...,1. The left-hand side of (11.91) re- 
duces to (FY (DHn;q,t),hgen—a), the g,t-Schréder. If read(P) is an a, B-shuffle, we 
must have all the elements of o occurring at the top of columns. It is easy to see that 
none of these cars can form dinv-pairs with any other car that occurs before them in 
the reading order. We can identify columns containing elements of a at the top with 
selections in the right-hand side of (11.89) of the d-column tops to be regarded as 
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D steps, and one finds in fact that the Schréder dinv-statistic in the right-hand side of 
(11.89) is the same as dinv(P). 
For a subset D of {1,2,...,n— 1}, let 


DiplZ) = y Za, Xa °° * Zan 


a Sag <--< an 
Q=aqi4., > i¢D 


denote Gessel’s fundamental quasisymmetric function. Another equivalent form 
of the shuffle conjecture appearing in [37] is the statement that 


F (DHn; qt) =), gow) poe. ) Dn Jdes(read(P)) (x1 gree Xn) (11.92) 
PEP y 


where recall that for any permutation o € S;,, Ides(o) is the set of all i for which i+ 1 
occurs before i in o. The authors show that the sum 


Axz(™1, tee neg) = y gi) D,, raes(read(Py) (X15 tee Xn), (11.93) 
PEP» 
T fixed 
obtained by restricting the right-hand side of (11.92) to a fixed path 7, is a special 
case of a family of symmetric functions introduced by Lascoux, Leclerc, and Thibon 
[46] commonly called LLT polynomials. In [46] it is conjectured that these polyno- 
mials are always Schur positive, and two recent preprints claim to give independent 
proofs of this conjecture. One of these is by Grojnowski and Haiman [32], and uses 
the representation theory of Hecke algebras. The other, by S. Assaf [11], is a purely 
combinatorial construction involving objects called dual equivalence graphs. 

In [39, Theorem 6.8, p. 98] it is shown that if 7 is a path with the property that 
each non-empty column has its base on the diagonal x = y (a so-called balanced 
path), then the Schur coefficients of Az (x1,...,%n;q) can be expressed in terms of 
the charge statistic of Algorithm 11.2.26. Hence one could hope that for general LLT 
polynomials, or at least for the type of LLT’s corresponding to Dyck paths, there is 
some similar formula. Note that the product eg 7) in (11.63) can be easily expanded 
in terms of Schur functions, and in fact the cocneet of sg in eg is Ky: g. Thus there 
may be a way of associating powers of g with SSYT, depending on the shape of z in 
some way, to generate the LLT polynomial Az(x1,...,%3¢)- 


Problem 11.5.13 Find a nice formula, perhaps in terms of a generalized charge 
Statistic, for the coefficients in the Schur expansion of the LLT polynomial corre- 
sponding to a given Dyck path, i.e., the right-hand side of (11.93). 


Assaf’s construction in terms of dual equivalence graphs does yield a combinato- 
rial formula of sorts for an arbitrary LLT polynomial, but her construction is rather 
involved, and it is not yet known how to reproduce explicit formulas, like the one 
for balanced paths in terms of charge, from her formula. Note that the shuffle con- 
jecture gives a prediction for the coefficient of a monomial symmetric function in 
F (DH;;q,t), but does not (except in the hook case) give a nice conjecture for the 
more desirable Schur coefficients. 
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11.6 Rational Catalan combinatorics 
11.6.1 The superpolynomial invariant of a torus knot 


Throughout this section (m,n) is a fixed pair of relatively prime, positive integers. In 
the last few years an exciting generalization of the g,t-Schroder and the shuffle con- 
jecture has been introduced. This generalization depends on an arbitrary pair (m,n), 
and has an interpretation in terms of knot theory. By a knot we mean an embed- 
ding of a circle in R*, and by a knot invariant a polynomial that is the same on 
equivalent knots. Two classical knot invariants on a knot K are the Jones polynomial 
V(t) and the HOMELY polynomial Px (a,q). Dunfield, Gukov, and Rasmussen [17] 
hypothesized the existence of a superpolynomial knot invariant Px(a,q,t), which 
would contain the HOMFLY and Jones polynomials as limiting cases, as well as 
having other desirable properties. Possible definitions of the superpolynomial for 
torus knots 7(,,,) have recently been suggested by Angnanovic and Shakirov [1] (see 
also [2]), Cherednik [16], and Oblomkov, Rasmussen, and Shende [55]. All three 
methods seem to give the same polynomial, and in fact Gorsky and Negut [31] have 
proved the descriptions in [1] and [16] do in fact give the same polynomial. The de- 
scription in [1] is in terms of Macdonald polynomials, and Gorsky first realized that 
when m = n+ 1, if we use the Cherednik parametrization, then the superpolynomial 
can be expressed as the function C,,(g,t,—a) from (11.89), giving a completely new 
interpretation for the g, t-Schréder. 

In [55] a conjectured combinatorial expression for the superpolynomial of 
T(m,n) in terms of weighted lattice paths is given, which we now describe. Let 
Grid(m,n) be the n x m grid of labeled squares whose upper-left-hand corner square 
is labeled with (n — 1)(m— 1) —1, and whose labels decrease by m as you go down 
columns and by 7 as you go across rows. For example, 


[li] 4] -3 
[8 | 1 | -6 
[5 [| —2[ -9 
Grid(3,7)=|2_| —5 | —12 
-1| =8 | =15 
| —4 | —11 | —18 
| -7 | -14 | -21 


To the corners of the squares of Grid(m,n) we associate Cartesian coordinates, where 
the lower-left-hand corner of the grid has coordinates (0,0), and the upper-right-hand 
comer of the grid (m,n). Let Le x) denote the set of lattice paths 7 for which none 
of the squares with negative labels are above 7. (This agrees with our definition of 
ae from Section 11.2.2 as paths that stay above the line my = nx). For a given 7, 
we let area(z) denote the number of squares in Grid(m,n) with positive labels that 
are below a. Furthermore, let dinv(z) denote the number of squares in Grid(m,n) 
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that are above z and whose arm and leg lengths satisfy 


a eos 
l+1 n l 


(11.94) 


For example, if (m,n) = (3,7) and t =NNNNNEENNE, then area(z) = 2 (corre- 
sponding to the squares with labels 2 and 5). Also, dinv(z) = 2; the squares with 
labels 11,8,4,1 havea =1=1,a=1,/=0,a=0,!/=1,a=/=0, respectively, and 
so the squares with labels 8 and 11 do not satisfy (11.94), while the squares with 
labels 1 and 4 do. 

Next we define a generalization of the formula (11.89) for general (m,n). Given 
TE Linn)? let R(z) denote the set of labels of squares that are at the top of some 
column of 7. Say these labels occur in columns cj,c2,...,cx, as we move left to 
right. Then for 1 <i<k, let t; denote the label of the square that is in the same 
row as the square at the top of column c;, and also in column cj+1, and set T(z) = 
{ti,t2,..-,tz-1}. For example, if z is the path on the left of Figure 11.22, then 


R(x) = {—-3,1,5} (11.95) 
T (x) = {—6,—2}. 
Now form a vector (7) = (Q@%,...,Q) consisting of the elements of R(z) in de- 


creasing order, and let c;() denote the number of elements of R(z) that are larger 
than @;, minus the number of elements of T (7) that are larger than a. For the exam- 
ple of (11.95), we have a = (5,1,—3), and so cj = 0-0 =0,c2 = 1-0 = 1, c3 
2—1=1. Furthermore set co = —1. 


Conjecture 11.6.1 [55] For any pair (m,n) of positive, relatively prime integers, 


Prim (—wat) = Y gti Mere TT (14+w/q*). (11.96) 
meLinn cee 


Here T (m,n) is the (m,n) torus knot, which winds around the torus m times in one 
direction and n times in the other before returning to the starting point, and we use 
the parametrization of the superpolynomial occurring in [16, p. 18, eq. (2.12)]. 


Exercise 11.6.2. Show that ifm =n + 1, (11.96) reduces to (11.89). 


There is also a version of the shuffle conjecture for any (m,n). Let an (m,n)- 
parking function be a path m7 € Le») together with a placement of the integers | 
through n (called cars) just to the right of the N steps of 7, with strict decrease down 
columns. For such a pair P, we let rank(/) be the label of the square that contains j, 


and we set 
tdinv(P) = |{(i, 7): 1 <i< j <mand rank(i) < rank(j) < rank(i) + m}/|. 


Furthermore we let the reading word read(P) be the permutation obtained by listing 
the cars by decreasing order of their ranks. For the (3,7)-parking function of Figure 
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11.21, tdinv = 3, with inversion pairs formed by pairs of cars (6,7), (4,6), and (2,4), 
and the reading word is 7642531. Let maxtdinv(z) be tdinv of the parking function 
for a whose reading word is the reverse of the identity, and for any parking function 
P for 7 set 


dinv(P) = dinv(z) + tdinv(P) — maxtdinv(z). 
Then the combinatorial side of the (m,n) shuffle conjecture is the function 


di 
Bimn) (x1, +++5Xn3, t) a y q Ba EN ERE) acta (x1, eke Xn). 
(m,n) parking functions P 


(11.97) 


Piw;];BRin 


Figure 11.21 
A (3,7)-parking function. 


Gorsky and Negut [31] show how the results of Aganagic and Shakirov on torus 
knot invariants can be expressed in terms of Macdonald polynomials using advanced 
objects such as the Hilbert scheme. Bergeron, Garsia, Leven, and Xin [12, 13] have 
shown how this Macdonald polynomial construction can be done combinatorially 
with plethystic symmetric function operators, and in fact they define operators Qi, ») 
for any relatively prime (m,n) by a recursive procedure. The rational shuffle con- 
jecture can then be phrased as Q(m,»)(—1)" = Bonn) (%1,--+,4n3,t). The symmetric 
function Q(n+1 n) (—1)" reduces to Ve, and so the rational shuffle conjecture reduces 
to the original shuffle conjecture when m=n-+1. 

Gorsky and Negut also conjecture that Qn) (—1)" is the Frobenius series of the 
unique finite-dimensional irreducible representation of the rational Cherednik alge- 
bra with parameter m/n, with respect to a certain bigrading. Hikita [43] has shown 
that Bom n) (x1,---;n3q,t) is the bigraded Frobenius series of certain S,,-modules aris- 
ing in the study of the homology of type A affine Springer fibers, which gives another 
possible way of attacking the rational shuffle conjecture. We mention that the portion 
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of the right-hand side of (11.97) corresponding to a fixed path 7 is an LLT polyno- 
mial, and hence is Schur positive. 

Many of the results in this chapter involving the special cases t = 1 and t = 1/q 
have elegant extensions to general, relatively prime (m,n). For example, Gorsky and 
Negut [31] prove that when ¢ = 1/q the q,t-Catalan for (m,n), obtained by taking the 
coefficient of 51» in Q(m,,)(—1)", reduces to 


mil’ 


Another example is that the total number of (m,n)-parking functions is m’~!. 


11.6.2 Tesler matrices and the superpolynomial 


There is a general formula for Q(,, ») (—1)" in terms of Tesler matrices. 


Theorem 11.6.3 (Gorsky, Negut 2013) For any pair of positive, relatively prime in- 
tegers (m,n), 


Omn(-IP= YO [Jee T] (leniei + Uae -leiitiler) TJ] (-levilas- 


Cé€Tes(m,n) i=1 1<i<m 2<i+l<j<m 
Cii41>0 ci,j>0 
(11.98) 
Here Tes(m,n) is the set of m x m upper-triangular matrices C satisfying 
in i—I1)n 
cuit Veig—- Vega AB 2 1<i<m. (11.99) 
j>i j<i m m 


Example 11.6.4 Given a symmetric function f expressed as a polynomial in the ex, 
the coefficient of sy» in f can be found by simply replacing each ex by 1. Now when 
m=n-+ | the conditions (11.99) reduce to the hook sums all equal | except for the 
first, which equals 0. Having a first hook sum equal to 0 forces the first row to be all 
zeros, and so Tes(n+ 1,n) is really just the same as Tes(n). Hence taking the weights 
from (11.98) when n = 3 for the matrices in (11.77), setting all e.,, = 1, we get 


C3(q,t) =1+ ([2] —[1]) + (2] - [1]) + (2) - [)74 
+ ([3] — [2])([2] — [1]) -M@ — M([2] — [1]) 
=14+(¢+t-1)+(¢+t—1)+ (¢+t-1)* 
+(¢ +qt+t?—q-t))(q+t-1) 

(1—q)(1—#)-(1—@)(1—-#)(¢+t-1) 


=Gt¢ttgqtt+ge +e. 


Garsia and Haglund [24] independently obtained a Tesler matrix expression for Vey, 
although it is a bit more complicated to state than the m = n-+ 1 case of (11.98). 


746 Handbook of Enumerative Combinatorics 


The superpolynomial for the (m,n) Torus knot can be defined analytically as 


Prin) (a,q,t) = y (—a)"~*(Oomn)(—1)", edhtn—a)- 
d>0 


As a corollary of (11.98), Gorsky and Negut obtain the following. 


Corollary 11.6.5 For any pair of positive, relatively prime integers (m,n), we have 


Primn) (a,q,t) = y II (1 —a) II ([cii+1 1 Vos = [ciz+1]}q1) 


CEéTes(m,n) 1<i<m l<i<m 
cii>0 
(—M lcijlg- 
2<i+1l<j<m 
cj,j>0 


There are a number of intriguing open problems involving the combinatorics of 
(m,n)-Catalan paths. For example, there is a candidate extension of the zeta map of 
Figure 11.8 that can be described as follows. Given a path 7 € Linn)? call the set of 
comers of grid squares that are touched by z the “vertices” of 2. Next define S(z) 
to be the set consisting of the labels of those squares whose upper-left-hand corners 
are vertices of 2. A given label in S(7) is called an N label if the vertex associated to 
it is the start of an N step, otherwise it is called an E label. For example, if 7 is the 
path on the left in Figure 11.22, then 


m=NNNNNENENE __ S(x) = {-10,—7,—4,—1,2,5,—2, 1, -6,-3}. (11.100) 
We now define the “sweep map” of [6], denoted ¢, from Lenn) to ae 
lows: Order the elements of S(z) in increasing order to create a vector of labels 
D(x) = (d1,d2,...,dm+4n). Then create a path @(2) by defining the ith step of (2) 
to be an N step if d; is an N label, and an E step if d; is an E label. For the example 
in (11.100), we have 


) as fol- 


D(x) = (—10,—7, -6,—4, -3,—-2,-1,1,2,5) (a) =NNNNENNENE. 


+4 
(m,n) 


Exercise 11.6.6 Show that when m=n-+ 1, paths in L are in bijection with paths 


in Lian) and that the sweep map reduces to the € map of Figure 11.8. 


Problem 11.6.7 Prove that for general coprime (m,n) the sweep map is a bijection 


+ + 
from Linn) > Linn)* 


This problem has been studied by Gorsky, Mazin, and Vazirani [30] and Armstrong, 
Loehr, and Warrington [6]. See also [8]. In [30] it is shown that the sweep map is a 
bijection whenever m = kn+ 1 or m= kn — | for some positive integer k. We note 
that in the case m = kn + 1 Loehr [49, 51] (see also [39, pp. 108-109]) has defined 
an extension of the bounce statistic, which when combined with area generates the 
q,t-Catalan for m = kn + 1. For this “paths in a n x kn rectangle” case there is also 
an interpretation for the rational shuffle conjecture in terms of a generalization of 
diagonal harmonics (see [37]). 
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Figure 11.22 
The sweep map. 
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12.1 Introduction 


Hints of the study of patterns in permutations date back a century, to Volume I, Sec- 
tion III, Chapter V of MacMahon’s 1915 magnum opus Combinatory Analysis [126]. 
In that work, MacMahon showed that the permutations that can be partitioned into 
two decreasing subsequences (in other words, the 123-avoiding permutations) are 
counted by the Catalan numbers. Twenty years later, Erdés and Szekeres [84] proved 
that every permutation of length at least (k—1)(€—1)+ 1 must contain either 12---k 
or £---21. Roughly twenty-five years after Erd6s and Szekeres, Schensted’s famous 
paper [146] on increasing and decreasing subsequences was published. 

Most, however, date the study of permutation classes to 1968, when Knuth pub- 
lished Volume 1 of The Art of Computer Programming [118]. In Section 2.2.1 of 
that book, Knuth introduced sorting with stacks and double-ended queues (deques), 
which leads naturally to the notion of permutation patterns. In particular, Knuth ob- 
served that a permutation can be sorted by a stack if and only if it avoids 231 and 
showed that these permutations are also counted by the Catalan numbers. He inspired 
many subsequent papers, including those of Even and Itai [85] in 1971, Tarjan [156] 
in 1972, Pratt [140] in 1973, Rotem [145] in 1975, and Rogers [144] in 1978. 

Near the end of his paper, Pratt wrote that 


From an abstract point of view, the [containment order] on permutations 
is even more interesting than the networks we were characterizing. This 
relation seems to be the only partial order on permutations that arises in 
a simple and natural way, yet it has received essentially no attention to 
date. 


Pratt’s suggestion to study this order in the abstract was taken up a dozen years later 
by Simion and Schmidt in their seminal 1985 paper “Restricted permutations” [148]. 
The field has continually expanded since then, and is now the topic of the conference 
Permutation Patterns, held each year since its inauguration (by Albert and Atkinson) 
at the University of Otago in 2003. 

Several overviews of the field have been published, including Kitaev’s 494-page 
compendium Patterns in Permutations and Words [112], one chapter in B6na’s under- 
graduate textbook A Walk Through Combinatorics [54] and several in his monograph 
Combinatorics of Permutations [50], and Steingrimsson’s survey article [155] for the 
2013 British Combinatorial Conference. In addition, the proceedings of the confer- 
ence Permutation Patterns 2007 [125] contains surveys by Albert [2], Atkinson [22], 
Bona [53], Brignall [62], Kitaev [111], Klazar [117], and Steingrimsson [154] on 
various aspects of the field. 

This survey differs significantly from prior overviews. This is partly because 
there is a lot of new material to discuss. In particular, Section 12.2.5 presents Fox’s 
results on growth rates of principal classes. After that, Section 12.3 is peppered with 
recent results, while Section 12.4 presents some new results improving on those pub- 
lished. More significantly, this survey differentiates itself from previous summaries 
of the area by its focus on permutation classes in general. 
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Figure 12.1 
The containment order on permutations. 


In order to maintain this focus, a great many beautiful results have been omit- 
ted. Thus, despite the impressive results of Elizalde [83], consecutive patterns will 
not be discussed. Nor will there be any discussion of mesh patterns, which began 
as a generalization of the “generalized” (now called vincular) patterns introduced by 
Babson and Steingrimsson [28] in their classification of Mahonian statistics but have 
since been shown to be worthy of study on their own via the wonderful Reciprocity 
Theorem of Braéndén and Claesson [60]. We similarly neglect two questions raised 
by Wilf in [165]: packing densities (which Presutti and Stromquist [141] have shown 
can be incredibly interesting) and the topology of the poset of permutations (where 
McNamara and Steingrimsson [130] have established some significant results). In- 
deed, we even ignore the original application to sorting, despite the deep results of 
Albert and Bousquet-Mélou [10] and Pierrot and Rossin [137, 138]. Alas, even this 
list of omitted topics contains omissions, for which I apologize. 


12.1.1 Basics 


Throughout this survey we think of permutations in one-line notation, so a permu- 
tation of length 1 is simply an ordering of the set [1,n] = {1,2,...,n} of integers. 
The permutation 2 contains the permutation o of length k if it has a subsequence of 
length k that is order isomorphic to o, i.e., that has the same pairwise comparisons 
as oO. For example, the subsequence 38514 is order isomorphic to 25413, so 25413 
is contained in the permutation 36285714. Permutation containment is perhaps best 
seen by drawing the plot of a permutation, which is the set of points {(i,(i))} as 
shown on the right of Figure 12.1. Permutation containment is a partial order on the 
set of all finite permutations, so if o is contained in z we write o < 2. If o £ 7, we 
say that 7 avoids o. 

The central objects of study in this survey are permutation classes, which are 
downsets (a.k.a., lower order ideals) of permutations under the containment order. 
Thus if @ is a class containing the permutation z and o < z then o must also lie in 
@. Given any set X of permutations, one way to obtain a permutation class is to take 
the downward closure of X, 


Sub(X) ={o : o <a for some 7 € X}. 


There are many other ways to specify a permutation class, for example as the set of 
permutations sortable by a particular machine, as the set of permutations that can be 
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“drawn on’ a figure in the plane (considered at the end of this subsection), or by a 
number of other constructions described in Section 12.3. However, by far the most 
common way to define a permutation class is by avoidance: 


Av(B) = {a : m avoids all B € B}. 


If one permutation of B is contained in another then we may remove the larger one 
without changing the class. Thus we may take B to be an antichain, meaning that no 
element of B contains any others. In the case that B is an antichain we call it the basis 
of this class. The case where B is a singleton has received considerable attention; we 
call such classes principal. 

The pictorial view shown in Figure 12.1 makes it clear that the containment or- 
der has the eight symmetries of the square, which (from the permutation viewpoint) 
are generated by inverse and reverse. For example, the classes Av(132), Av(213), 
Av(231), and Av(312) are all symmetric (isomorphic as partially ordered sets), as 
are the classes Av(123) and Av(321). This shows that there are only two essentially 
different principal classes avoiding a permutation of length 3. There are seven essen- 
tially different principal classes avoiding a permutation of length 4. 

We are frequently interested in the enumeration of permutation classes. Thus 
letting @, denote the set of permutations of length n in the class @, we wish to 
determine (either exactly or asymptotically) the behavior of the sequence |% |, |@|, 
... (this sequence is called the speed of the class in some contexts). One way of 
doing this is to explicitly compute the generating function of the class, 


y |Galx” = y xl", 


n>0 NEO 


where here |7| denotes the length of 2. We are often interested in whether this gen- 
erating function is rational (the quotient of two polynomials), algebraic (meaning 
that there is a polynomial p(x,y) € Q[x,y] such that p(x, f(x)) = 0), or D-finite (if 
its derivatives span a finite dimensional vector space over Q(x)). 

In practice it is often quite difficult to compute generating functions of permuta- 
tion classes, and thus we must content ourselves with the rough asymptotics of |@;,|- 
To do so we define the upper and lower growth rate of the class @ by 


er(@) =limsup V/|@,,| and gr(@) = lim inf Vv |Gal; 
n—yoo ox Ueno 
respectively. It is not known if these two quantities agree in general. 
Conjecture 12.1.1 For every permutation class @, gt(@) = gr(@). 


If the upper and lower growth rates of a class are equal, we refer to their common 
value as the (proper) growth rate of the class. In order to establish a sufficient con- 
dition for the existence of proper growth rates, we need two definitions. Pictorially, 
the (direct) sum of the permutations z and o, denoted by 2 Go, is shown on the left 
of Figure 12.2. If z has length k and o has length @, we can also define the sum of 7 
and o by 

; (i fori € [1,k], 
(T@e)G) = { Ae k)+k forie heen 


The analogous operation depicted on the right of Figure 12.2 is called the skew sum. 
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TGO= TOO0= 


Figure 12.2 
The sum and skew sum operations. 


The permutation class @ is said to be sum closed (respectively, skew closed) if 
TOO € G (respectively, 7G o € @) for every pair of permutations o,z € @. The 
permutation 7 is further said to be sum (respectively, skew) decomposable if it can 
be expressed as a nontrivial sum (respectively, skew sum) of permutations, and sum 
(respectively, skew) indecomposable otherwise. It is easy to establish that a class 
is sum (respectively, skew) closed if and only if all of its basis elements are sum 
(respectively, skew) indecomposable. By observing that a single permutation cannot 
be both sum and skew decomposable, we obtain the following. 


Observation 12.1.2 Every principal permutation class is either sum or skew closed. 


The sequence {a,} is said to be supermultiplicative if am+4, > Gmd@y for all m 
and n. Fekete’s Lemma states that if the sequence {a,} is supermultiplicative then 
lim /a, exists and is equal to sup ¥/a,. A simple application of this lemma gives us 


the following result. 


Proposition 12.1.3 (Arratia [19]) Every sum closed (or, by symmetry, skew closed) 
permutation class has a (possibly infinite) growth rate. In particular, this holds for 
every principal class. 


Proof. Suppose @ is a sum closed permutation class, and thus 7® 0 € Ginin for 
all z € G, and o € G,. Moreover, a given T € Gjn1, arises in this way from at 
most one such pair so |G@m+n| > |Gn||Gn|. This shows that the sequence {|@,|} is 
supermultiplicative, and thus lim ¢’/|@;,| exists by Fekete’s Lemma. | 


In particular, gr(@) > %/|@,,| for all sum or skew closed classes @ and all integers 
n, which can be used to establish lower bounds for growth rates of these classes. (We 
appeal to this fact in the proofs of both Proposition 12.2.1 and Theorem 12.2.10.) 

Continuing our exploration of the sum operation, for every permutation 7 there 
are unique sum indecomposable permutations 0,...,Q% (called the sum compo- 
nents of 7) such that 7 = a, ®---@ a. Therefore the permutations in a sum closed 
class can be viewed as sequences of sum indecomposable permutations. In particular, 
if a class is sum closed and the generating function for its nonempty sum indecom- 
posable members is s, then the generating function for the class is 1/(1 —s). 

A permutation is layered if it is the sum of decreasing permutations. Clearly 
every decreasing permutation is sum indecomposable, so by our previous remarks 
the generating function for the class of layered permutations is 


1 1—-x 


[eee eed 
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(We could have instead observed that the layered permutations are in bijection with 
integer compositions.) It is not difficult to show that the basis of the class of layered 
permutations is {231,312}. 

A permutation is separable if it can be built from the permutation | by repeated 
sums and skew sums. For example, the permutation 576984132 is separable: 


765984132 = 32154010132 
= (321621)616(1621) 
(16161) @(161))e1e(1e(161)). 


The term separable is due to Bose, Buss, and Lubiw [56], who proved that the sep- 
arable permutations are Av(2413,3142), although these permutations first appeared 
in the much earlier work of Avis and Newborn [27]. 

Recall that the little Schréder number indexed by n— 1 counts the number of 
ways to insert parenthesis into a sequence of m symbols in such a way that every pair 
of parentheses surrounds at least two symbols or parenthesized groups and no paren- 
theses surround the entire sequence (see Stanley [152, Exercise 6.39.a]), while the 
large Schroder numbers are twice the little Schroder numbers. It follows immedi- 
ately from our decomposition above that the separable permutations are counted by 
the large Schroder numbers. For example, the permutation from our example can be 


encoded by the pair 
©, ((ee@)(0e)) 0 (o(ee)), 


where © indicates that the outermost division is a skew sum and each pair of paren- 
theses within the expression denotes the opposite type of decomposition as the pair 
enclosing it. These numbers begin 1, 2, 6, 22, 90, 394, 1806, ..., and have the alge- 


braic generating function 
3-—x-—V1—6x+x? 
5 : 


It follows that the growth rate of the separable permutations is approximately 5.83. 

The pictorial perspective presented in Figure 12.1 leads naturally to a geometric 
treatment of the containment order, in which permutations are objects of a moduli 
space. From this viewpoint, the fundamental objects are figures, which are simply 
subsets of the plane. Given two figures &,'Y C R’, the figure ® is involved in the 
figure ‘VY, denoted ® < V if there are subsets A,B C R and increasing injections 
T : A— Rand t, : B— R such that 


® CA x Band t(®) CY, 


where T(®) = {(t%(a),%(b)) : (a,b) € PB}. 

The involvement order is a preorder on the collection of all figures (it is reflexive 
and transitive but not necessarily antisymmetric). If 6 < Y and V < O, then we say 
that ® and Y are equivalent figures and write ® ~ V. If two figures have only 
finitely many points, it can be shown that they are equivalent if and only if one can 
be transformed to the other by stretching and shrinking the axes. Figure 12.3 shows 
two examples of equivalent figures. 
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Figure 12.3 


Two pairs of equivalent figures. 


Figure 12.4 
On the left, the permutation 63124578 can be drawn on a V. On the right, the permu- 
tation 18756432 can be drawn on an X. 


We call a figure independent if no two points of the figure lie on a common 
horizontal or vertical line. It is clear that every finite independent figure is equivalent 
to the plot of a unique permutation (which we could call its modulus if adopting 
the moduli space perspective). Under this identification, the involvement order on 
equivalence classes of finite independent figures is isomorphic to the containment 
order on permutations. Every figure ® C R? therefore defines a permutation class, 


Sub(®) = {z : wis equivalent to a finite independent figure involved in ®}, 


which we call a figure class. If 7 € Sub(®) then we say that 7 can be drawn on 
the figure ®. Two examples of this are shown in Figure 12.4. For example, the class 
Sub(V) of permutations that can be drawn on the leftmost figure in Figure 12.3 con- 
tains all permutations that consist of a decreasing sequence followed by an increas- 
ing subsequence. It is not difficult to show that Sub(V) = Av(132,231). The class 
Sub(X) was studied by Elizalde [82], who established a bijection between this class 
and a set of permutations studied by Knuth in Volume 3 of The Art of Computer 
Programming [119]. Both of these classes are examples of geometric grid classes, 
introduced in Section 12.4.2. 


12.1.2 Avoiding a permutation of length three 


Here we briefly consider the two simplest nontrivial principal classes: Av(231) and 
Av(321). As mentioned in the introduction, both of these classes are counted by 
the Catalan numbers (and thus have growth rates of 4), so there ought to be nice 
bijections between them and Dyck paths. Indeed there are, as shown in Figure 12.5, 
and it is almost the same bijection for both classes. 

The bijection depicted in Figure 12.5 is originally due to Knuth [118, 119], 
though it was Krattenthaler [121] who gave the nonrecursive formulation we de- 
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Figure 12.5 
The construction of Dyck paths from permutations avoiding 231 (left) and 321 


(right). 


empty avoids 231 


avoids 231 empty 


Figure 12.6 
A 231-avoiding permutation. 


scribe. In this bijection, we plot a permutation and then draw the Dyck path that 
lies just below its right-to-left minima (an entry is a right-to-left minimum if it is 
smaller than every entry to its right). It is then possible to show that the positions 
of the non-right-to-left minima can be determined from this Dyck path. There are at 
least eight more essentially different bijections between these two classes. For the 
rest of them we refer to Claesson and Kitaev’s survey [77]. 

Despite the elegance of this bijection, Av(231) and Av(321) are very different 
classes. Indeed, Miner and Pak [131] make a compelling argument that there is no 
truly “ultimate” bijection between these two classes. As discussed in Section 12.3, 
Av(231) has only countably many subclasses, while Av(321) has uncountably many. 
Indeed, Albert and Atkinson [3] have proved that every proper subclass of Av(231) 
has a rational generating function, while a simple counting argument shows that 
Av(321) contains subclasses whose generating functions aren’t even D-finite. 

Another way to see the contrast between Av(231) and Av(321) is to consider the 
decomposition of a 231-permutation shown in Figure 12.6. As this figure shows, all 
entries to the left of the maximum in a 231-avoiding permutation must lie below all 
entries to the right of the maximum. Going in the other direction, every permutation 
constructed in this manner avoids 231. Thus if we let f denote the generating func- 
tion for the 231-avoiding permutations (including the empty permutation), we see 
immediately that 

feaxf +l, 


so f is the generating function for the Catalan numbers. 
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Figure 12.7 
On the left, a drawing of the permutation 257193468 on two parallel lines. On the 
right, three parallel lines. 


Despite the similarities of Figure 12.5, there is no analogue of Figure 12.6 for the 
321-avoiding permutations. Perhaps the best structural description of this class one 
can give is the following. 


Proposition 12.1.4 The class of permutations that can be expressed as the union of 
two increasing subsequences is Av(321). 


Proof. Label each entry of z € Av(321) by the length of the longest decreasing 
subsequence that ends at that entry. Clearly we have only used the labels 1 and 2, and 
for each value of i the entries labeled by i form an increasing subsequence. a 


In his thesis, Waton proved the following geometric version of Proposition 12.1.4 
(illustrated on the left of Figure 12.7). 


Proposition 12.1.5 (Waton [163]) The class of permutations that can be drawn on 
any two parallel lines of positive slope is Av(321). 


Our proof of Proposition 12.1.4 generalizes easily to show that every k---21- 
avoiding permutation can be expressed as the union of k — | increasing subsequences. 
However, Waton showed that Proposition 12.1.5 does not extend in the same way. 
Consider three parallel lines at distances d, and dz from each other as depicted on 
the right of Figure 12.7. Clearly every permutation that can be drawn on these lines 
avoids 4321, but Waton proved that every different choice of the ratio d, /d leads to 
a different proper subclass of Av(4321). 


12.1.3. Wilf-equivalence 


As we saw in the last subsection, the permutations 231 and 321 are equally avoided. 
Instead of being a coincidence of small numbers, this turns out to be a common phe- 
nomenon. The classes @ and Y are said to be Wilf-equivalent if they are equinu- 
merous, i.e., if || = |F,| for every n. In this subsection we are primarily concerned 
with Wilf-equivalence of principal classes, so instead of saying that Av(f) and Av(7) 
are Wilf-equivalent we simply say that B and y are Wilf-equivalent. 
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Large swaths of Wilf-equivalences can be explained via a much stronger notion of 
equivalence, though doing so requires a shift of perspective to full rook placements 
(frps). These consist of a Ferrers board with a designated set of cells, called rooks 
(here drawn as dots), so that each row and column contains precisely one rook. For 
example, both objects in Figure 12.8 are frps. (We use French/Cartesian indexing 
throughout this survey, so for us a Ferrers board is a left-justified array of cells in 
which the number of cells in each row is at least the number of cells in the row 
above.) 

There is a natural partial order on the set of all frps: given frps R and S, we say 
that R is contained in S if R can be obtained from S by deleting rows and columns. 
Furthermore, this partial order generalizes the containment order on permutations. 
To make this precise, let us call a frp square if the underlying Ferrers board is square 
and say that the permutation z of length n corresponds to the n x n frp with rooks in 
the cells (i, 2(i)) for every i. When restricted to square frps, the containment order 
on frps is equivalent to the containment order on the corresponding permutations. 

Because of this correspondence between permutations and square frps, we say 
that a frp contains the permutation o if it contains the square frp corresponding to o 
and otherwise say that it avoids 0. Observe that the entire square containing o must 
be contained within the frp; for example, the frp below contains 12 but avoids 21. 


We say that the permutations B and y are shape-Wilf-equivalent if given any shape 
A, the number of B-avoiding frps of shape A is the same as the number of y-avoiding 
frps of shape 2. Shape-Wilf-equivalence implies Wilf-equivalence by considering 
only square shapes, but it also implies quite a bit more. 


Proposition 12.1.6 (Babson and West [29]) If B and y are shape-Wilf-equivalent, 
then for every permutation 6, B ® 6 and y@ 6 are also shape-Wilf-equivalent. 


Proof. Suppose that there is a bijection between B-avoiding and y-avoiding frps of 
every shape. Now fix a shape 7. We construct a bijection between B © 6-avoiding 
frps and y@ 6-avoiding frps of shape 7. Let R be a B @ 6-avoiding frp of shape 1. 
We call a cell of R dangerous if there is a copy of 6 completely contained in the 
region above and to the right of the cell. The entire set of dangerous cells is called the 
danger zone (see Figure 12.8). The danger zone forms a (possibly empty) Ferrers 
board nestled in the bottom-left corner of R. Ignoring the rookless rows and columns 
of the danger zone, we thus obtain a B-avoiding frp. We may then use the bijection 
between B-avoiding and y-avoiding frps of that shape to produce a y@ 6-avoiding 
frp of shape A, as desired. | 


Only two shape-Wilf-equivalence results are known. We sketch bijective proofs 
for both. 


Theorem 12.1.7 (Backelin, West, and Xin [30]) For every value of k, the permuta- 
tions k---21 and 12---k are shape-Wilf-equivalent. 
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Figure 12.8 
On the left, a frp with its danger zone for the permutation 12 shaded. On the right, 
the frp that the danger zone contracts to after removing rookless rows and columns. 
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0 0 0 0 0 0 0 0 0 


Figure 12.9 
Krattenthaler’s bijection, shown here converting a 321-avoiding frp to a 123-avoiding 


frp. 


By Proposition 12.1.6 this implies that the permutations k---21@B and 12---k® 
B are Wilf-equivalent for every permutation f. Special cases of Theorem 12.1.7 had 
been established before the general case above; West proved the k = 2 case in his 
thesis [164], while the k = 3 case was shown by Babson and West [29]. Kratten- 
thaler [122] was the first to give a truly nice bijection between k---21-avoiding and 
12---k-avoiding frps, using the growth diagrams of Fomin [87]. 

Consider the labeled frp shown on the left of Figure 12.9. In this diagram every 
corner is labeled by an integer partition (the labels lie slightly above and to the right 
of the corners). This partition (written in shortened notation) represents the shape of 
the resulting standard Young tableaux when the partial permutation lying below and 
to the left of that corner is passed through the Robinson—Schensted correspondence. 
As one can imagine, there are a variety of restrictions on such growth diagrams. 
For example, the labels along the northwest-southeast boundary form an oscillating 
tableaux, meaning that each partition differs from the previous partition by precisely 
one box. 
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We will not concern ourselves with the other requirements here, except to call 
labeled tableaux formed in this way valid. One of the facts that Krattenthaler es- 
tablished is that given the oscillating tableaux along the border of a valid growth 
diagram, one can reconstruct all of the interior corner labels, and from these, the po- 
sition of the rooks in the corresponding frp. Thus we need only concern ourselves 
with the labels of this border. It follows from Fomin’s work that by conjugating the 
partitions labeling the border of a valid growth diagram one obtains another valid 
growth diagram. 

Now recall that if a frp avoids k---21, its shape under Robinson—Schensted has 
fewer than k parts (Schensted’s Theorem [146]). Equivalently, all parts along the 
border of its valid growth diagram will have fewer than k parts. As Krattenthaler 
observed, this combination of Fomin’s growth diagrams and Schensted’s Theorem 
shows that if we conjugate each of those partitions, we obtain the border of a valid 
growth diagram in which no part is k or larger. Therefore we have produced a unique 
frp avoiding 12---k, giving a bijection between k...21-avoiding and 12 ---k-avoiding 
frps. Krattenthaler’s bijection not only proves Theorem 12.1.7 but also its analogue 
for involutions, first established by Bousquet-Mélou and Steingrimsson [59]. 

We move on to the second of two known examples of shape-Wilf-equivalence. 


Theorem 12.1.8 (Stankova and West [149]) The permutations 231 and 312 are 
shape-Wilf-equivalent. 


Motivated by Fomin’s growth diagrams, Bloom and Saracino [41] gave a beauti- 
ful bijective proof of Theorem 12.1.8. Consider a 231-avoiding frp, as shown in the 
upper-left diagram of Figure 12.10. This time, instead of partitions, we label each 
corner of the northwest-southeast border by the length of the longest increasing se- 
quence in the partial permutation lying below and to the left of that corner. We then 
read these labels to construct a labeled Dyck path, as shown on the right of this figure. 

We need a final term. A weak tunnel in a Dyck path is a horizontal segment 
between two vertices of the path that stays weakly below the path (the paths shown 
in Figure 12.10 each have a weak tunnel indicated in gray). Bloom and Saracino 
proved that these labeled Dyck paths satisfy three rules: 


e Monotone property: Labels increase by at most | after an up step and de- 
crease by at most | after a down step. 


e Zero property: The 0 labels are precisely those on the x-axis. 


e 231-avoiding tunnel property: Given two vertices at the same height con- 
nected by a weak tunnel, the label of the leftmost vertex is at most the label 
of the rightmost vertex. (This was called the “diagonal property” in [40].) 


They also proved that there is a bijection between labeled Dyck paths and 231- 
avoiding frps satisfying these three properties. Suppose we perform the same op- 
eration on 312-avoiding frps, as shown on the bottom of Figure 12.10. Bloom and 
Saracino showed that the resulting labeled Dyck paths have all of the same properties 
except for one; the 231-avoiding tunnel property becomes the 
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Figure 12.10 
Bloom and Saracino’s bijection between 23 1-avoiding (upper-left) and 312-avoiding 
frps (bottom-left). 


e 312-avoiding tunnel property: Given two vertices at the same height con- 
nected by a weak tunnel, the label of the rightmost vertex is at most the label 
of the leftmost vertex. 


To prove Theorem 12.1.8, we merely need to construct a bijection between labeled 
Dyck paths satisfying the monotone, zero, and 231-avoiding tunnel properties and 
those that satisfy the monotone, zero, and 312-avoiding tunnel properties. This (as is 
illustrated on the right of Figure 12.10) is done by fixing the labels on the x-axis at 
zero, and then replacing the label ¢ in the labeled Dyck path of a 231-avoiding frp by 
h—£+1, where h is the height of the corresponding vertex. 

As no other shape-Wilf-equivalences are known, we must pose the following 
question. 


Question 12.1.9 Do Theorems 12.1.7 and 12.1.8 (and their implications) constitute 
all of the shape-Wilf-equivalences ? 


To finish mapping the known world of Wilf-equivalence of principal classes, we 
need one more result. 


Theorem 12.1.10 (Stankova [150]) The permutations 1342 and 2413 are Wilf- 
equivalent, but not shape-Wilf-equivalent. 


Bloom [39] was the first to describe a nice bijection between these classes (more 
precisely, he constructed a bijection between 1423-avoiding permutations and 2413- 
avoiding permutations). By showing that his bijection preserves certain permutation 
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Figure 12.11 
A typical basis element for the class of permutations sortable by two increasing 
stacks in series. 


statistics, he was able to establish a conjecture of Dokos, Dwyer, Johnson, Sagan, 
and Selsor [80]. 

Finally, we remark that classes do not need to have the same number of basis 
elements to be Wilf-equivalent. For example, Atkinson, Murphy, and Ruskuc [24] 
showed that the class Av(1342) is Wilf-equivalent to the infinitely based class 


Av({2 (2k—1) 416385 --- (2k) (2k—3) : k>2}), 


which consists of those permutations that can be sorted by two increasing stacks in 
series (for a visualization of these basis elements, see Figure 12.11; a proof of this 
result using frps is given in Bloom and Vatter [42]). Burstein and Pantone [70] later 
showed that Av(1234) and Av(1324, 3416725) are Wilf-equivalent, along with other 
unbalanced Wilf-equivalences. 


12.1.4 Avoiding a longer permutation 


In the study of principal classes, there is a steep increase in difficulty when we in- 
crease the length of the basis element from 3 to 4. We begin this section with a 
routine bound, but end it with the easily-stated but notorious problem of computing 
the growth rate of Av(1324). 


Proposition 12.1.11 The growth rate of Av(k-+-21) is at most (k—1)?. 


Proof. Let m7 be ak---21-avoiding permutation of length n. By the obvious gen- 
eralization of Proposition 12.1.4 mentioned in Section 12.1.2, we can partition the 
entries of 7 into k — 1 increasing subsequences. Label these subsequences | to k— 1 
and extend this labeling to their entries. We can easily reconstruct 2 if we know the 
positions and values of the entries of each of these subsequences. We can record 
this information in two lists, one reading left to right, the other reading bottom to 
top. Thus there are at most (k — 1)?” permutations of length n that avoid k---21, as 
desired. a 


There does not seem to be an elementary way to establish a matching lower 
bound for Proposition 12.1.11. Béna [51, Corollary 5.8] proved that for every per- 
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Figure 12.12 

Demonstrating the Wilf-equivalence of a two-layer permutation and a monotone 
permutation of the same length. The first and last equivalences follow from Theo- 
rem 12.1.7, while the central equivalence is an application of symmetry. 


mutation B, 
gr(Av(B 621)) = 14+ Vgr(Av(B 6 1)) 


which implies that gr(Av(k---21)) = (k — 1), but his proof is quite involved. The 
first derivation of gr(Av(k---21)) follows from a much more general result of Regev. 


Theorem 12.1.12 (Regev [142]) The growth rate of Av(k-+-21) is (k—1)?. 


This immediately implies the following, which will be useful later when we 
bound the growth rate of principal classes avoiding layered permutations (Theo- 
rem 12.3.8). 


Corollary 12.1.13 For any layered permutation B of length k and consisting of at 
most two layers, the growth rate of Av(B) is (k—1)?. 


Proof. The single layer case is Regev’s Theorem 12.1.12, while the two-layer case 
follows by Theorem 12.1.7 and symmetry, as shown in Figure 12.12. | 


Gessel [95] later gave an explicit formula for the generating functions of the 
classes Av(k---21) in terms of determinants. Stanley [153] expressed the signifi- 
cance of Gessel’s result by writing that 


Gessel’s theorem reduces the theorems of Baik, Deift, and Johansson to 
“Just” analysis, viz., the Riemann-Hilbert problem in the theory of inte- 
grable systems, followed by the method of steepest descent to analyze 
the asymptotic behavior of integrable systems. 


(The quote refers to Baik, Deift, and Johansson’s proof [31] that, after appropriate 
rescaling, the distribution of the longest increasing subsequence statistic on permu- 
tations of length n converges to the Tracy-Widom distribution.) 

Gessel’s result was later rederived by Bousquet-Mélou [58] using the kernel 
method. In the case k = 4, she expressed the enumeration as an explicit sum, 


anl820)= Some hCG) Gri) (ie): 
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(Bousquet-Mélou had given an earlier derivation of this formula in [57].) The gen- 
erating function in the k = 4 case is known to be D-finite but nonalgebraic, and no 
“nice” formulas are known for any larger values of k. 

The Wilf-equivalences of Section 12.1.3 together with the trivial symmetries 
show that (from an enumerative perspective) there are only two more cases of prin- 
cipal classes avoiding a permutation of length 4: Av(1342) and Av(1324). Bona was 
the first to enumerate Av(1342), by constructing a bijection between the skew in- 
decomposable 1342-avoiding permutations and f (0, 1)-trees, which had been previ- 
ously counted by Tutte [158]. (Because the class Av(1342) is skew closed, its gen- 
erating function can be computed easily from the generating function for its skew 
indecomposables, as mentioned in Section 12.1.1.) 


Theorem 12.1.14 (Bona [46]) The generating function for Av(1342) is 


32x 
1 + 20x — 8x2 — (1 — 8x)3/2’ 


and thus the growth rate of this class is 8. 


Bloom and Elizalde [40] have recently shown how to derive this generating func- 
tion using the techniques of Bloom and Saracino [41] discussed in Section 12.1.3, 
which we briefly sketch. 

First, instead of counting Av(1342) they count the symmetric class Av(3124). A 
frp is board minimal if its rooks do not lie in any smaller Ferrers board, or equiva- 
lently, if it has a rook in each of its upper-right corners. There is an obvious bijection, 
which we denote by 7, between permutations and board minimal frps. It is also easy 
to see that were a frp from 7(Av(3124)) to contain 312, there would be a rook in 
an upper-right corner above and to the right of this copy of 312, a contradiction. 
Therefore these frps avoid 312. 

Next label the border of these frps as in Section 12.1.3 and convert them to 
Dyck paths. The corresponding Dyck paths must satisfy the monotone, zero, and 
312-avoiding tunnel properties, and in addition, because they arise from board min- 
imal frps, they satisfy the peak property: The labels rounding any peak (an up step 
followed by a down step) are @, 0+ 1, @ for some value of @. 

Thus we are reduced to counting the family of labeled Dyck paths satisfying the 
monotone, zero, 312-avoiding tunnel, and peak properties. Bloom and Elizalde were 
able to find the generating function for these by introducing a catalytic variable and 
applying Tutte’s quadratic method (which is described in Flajolet and Sedgewick [86, 
VIL 8.2]). 

This leaves a single case, that of Av(1324). As reported in [81], at the conference 
Permutation Patterns 2005, Zeilberger made the colorful claim that 


Not even God knows | Avjo90(1324)]. 
Steingrimsson is less pessimistic, writing in [155] that 


I’m not sure how good Zeilberger’s God is at math, but I believe that 
some humans will find this number in the not so distant future. 
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10 20 36 61 96 148 208 268 321 351 ... 
10 20 36 65 106 171 262 397 568 784 ... 
10 20 36 65 110 181 286 443 664 985 ... 
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Figure 12.13 
The number of 1324-avoiding permutations, grouped into columns by their number 
of inversions. 


Bona currently holds the record for the lowest upper bound on the growth rate of 
Av(1324). 


Theorem 12.1.15 (Bona [44]) The growth rate of Av(1324) is at most 13.74. 


B6na established Theorem 12.1.15 by refining the approach of Claesson, Jelinek, 
and Steingrimsson [76] presented in Section 12.3.1. Claesson, Jelinek, and Ste- 
ingrimsson made a further conjecture which, if true, would give a better upper bound. 
Consider the table shown in Figure 12.13. In this table, the entry in column k + 1 of 
row nis the number of 1324-avoiding permutations of length n with precisely k in- 
versions. As the gray cells are meant to indicate, it appears that the entries along a 
column are weakly increasing and eventually constant. They proved that the columns 
are eventually constant, but could not show that they are weakly increasing. If this 


conjecture could be proved, it would give the improved bound of er V2/3 < 13.01 
on the growth rate of Av(1324). Indeed this might be part of a much more general 
phenomenon; empirical evidence suggests that the analogous table for B-avoiding 
permutations has weakly increasing columns for all B 4 12---k. 

Arratia [19] had conjectured that gr(Av(B)) < (k— 1) for all permutations B of 
length k, and Bona [51] had gone even further, conjecturing that equality held if and 
only if B was layered. We know now (by Fox’s results presented in Section 12.2.5) 
that these conjectures are far from the truth, but the first known counterexample was 
the class Av(1324). Using quite a bit of computation and the insertion encoding 
(introduced by Albert, Linton, and RuSkuc [15]), Albert, Elder, Rechnitzer, Westcott, 
and Zabrocki [13] showed that 


gr(Av(1324)) > 9.47. 


Extrapolating from the amount their lower bounds improved as they increased the 
accuracy of their approximation, they guessed that the true value lies between 11 and 
12. Madras and Liu [127] later used Markov chain Monte Carlo methods to estimate 
that this growth rate lies between 10.71 and 11.83. 
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m(j) 


Figure 12.14 
The drawing on the left shows the Hasse diagram of the poset of a 1324-avoiding 
permutation. On the right, this Hasse diagram has been divided into three trees. 


At present, the best estimate seems to be due to Conway and Guttmann [78]. They 
extended the approach of Johansson and Nakamura [106] (who had computed the 
first 31 terms of the enumeration) to compute the first 36 terms of the enumeration of 
this class and then applied the methodology laid out in Guttmann [98] to approximate 
that 

| Av, (1324)| ~ Cu" v¥"n8, 


where C 9.5, v 0.04, g © —1.1. and u—the growth rate—is approximately 
11.60. 

Bevan [36] has recently established that gr(Av(1324)) > 9.81 by constructing a 
large subset of this class with a particularly regular structure. Sketching his proof 
would require quite a detour, but we attempt to convey a hint of his novel approach. 

Given a permutation 7, define the poset Pz on the points {(i,2(i))} in which 
(i, a(1)) < (j,a(J)) if both i < j and (i) < 2(j), ie., if this pair of entries forms a 
noninversion in z. The drawing on the left of Figure 12.14 shows the Hasse diagram 
of the poset of a 1324-avoiding permutation. Because this permutation avoids 1324, 
its Hasse diagram does not contain a diamond: 


ye 


Therefore if we restrict the Hasse diagram to 2(1) and all entries lying above and 
to its right we see a rooted tree, as drawn on the right of Figure 12.14. Next let 2(i) 
denote the greatest point lying below 2(1). If we restrict the Hasse diagram to 7(i) 
and all entries lying below and to its left, we see another rooted tree. Finally, for this 
example, if we let 2(j) denote the leftmost entry to the right of z(i) and restrict the 
Hasse diagram to this entry and all entries above and to its right, we see yet another 
rooted tree. Let us call rooted trees like the first and third up trees and trees like the 
second down trees. 
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Bevan’s construction consists of alternating up trees and down trees, while inter- 
leaving their vertices. However, this alone is not sufficient to avoid 1324. For exam- 
ple, the following interleaving of a down tree and an up tree contains 1324. 


alt 


Thus we must be careful in performing this interleaving. In constructing the permu- 
tation shown in Figure 12.14, we avoided creating a copy of 1324 by interleaving the 
vertices of the up trees with the principal subtrees of the down tree, as indicated by 
the shading in Figure 12.14 (a principal subtree of a rooted tree is a component of 
the forest obtained after removing the root). 


12.2. Growth rates of principal classes 


In the 1980s, Stanley and Wilf independently conjectured that every proper permuta- 
tion class (that is, every class except the class of all permutations) has a finite upper 
growth rate. Obviously, it suffices to prove this for principal classes. In this context, 
the growth rate of Av() is often (though not in this survey) called the Stanley—Wilf 
limit of B. 

Two noteworthy partial results were proved in the 20th century. First Bona [49] 
proved the Stanley—Wilf Conjecture for Av(B) when f is a layered permutation (we 
prove a stronger result in Theorem 12.3.8). Then Alon and Friedgut [18] used a result 
of Klazar [113] about Davenport-Schinzel sequences to prove that for every B there 
is a constant cg such that | Av,(B)| < ene where Y grows even more slowly than 
the inverse Ackermann function (the inverse Ackermann function itself is considered 
to be less than 5 for all “reasonable” values of n). This result caused Wilf’s belief to 
waver; in 2002 he [165] wrote that 


This conjecture had been considered a “sure thing,” but the results of 
Alon and Friedgut seem to make it somewhat less certain because a simi- 
lar bound, involving the Ackermann function, in the Davenport-Schinzel 
theory turns out to be best possible. 


In 2004, Marcus and Tardos [129] presented an elegant proof of the Stanley—Wilf 
Conjecture. About their proof, Zeilberger [168] wrote 


Once I thought that proofs of long-standing conjectures had to be diffi- 
cult. After all didn’t brilliant people make them, and didn’t these people, 
and many other brilliant people, unsuccessfully try to prove them? Isn’t 
that a meta-proof that the proof, if it exists, should be long, difficult, 
and ugly? And if not, shouldn’t they, at least, contain entirely new ideas 
and/or techniques? 
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We present Marcus and Tardos’ proof in Section 12.2.3. Of course, the proof of 
the Stanley—Wilf Conjecture only raises new questions. For example, what can be 
said about the relationship between the growth rate of Av(B) and the length of B? 
Many conjectures (some of which were mentioned in Section 12.1.4) have been made 
about this relationship, and most have been proven false. The first truly general result 
in this direction is due to Valtr, and appeared in a paper of Kaiser and Klazar [107]. 
We present a specialization. 


Proposition 12.2.1 For all sufficiently large k and all permutations B of length k, 
er(Av(B)) > k?/27. 


Proof. Setn = |k?/9| (the integer part of k7/9) and let B be any pattern of length 
k. Using linearity of expectation and Stirling’s Formula, we see that the probability 
that a permutation of length n chosen uniformly at random contains B is at most 


1 (n nk ai e\* 
=e < ; 
k! 6 < Ce? = of (k/e)* > (5 ) 
This quantity tends to 0 as k > ~ so for sufficiently large k at least half of the per- 


mutations of length n avoid B. Now Proposition 12.1.3 and Stirling’s Formula show 
that 


n 2 ! 2 
(ET k 


= Sk? (27 
V/n/2 9et/n/2 


gr(Av(B)) > ¥/|Avn(B)| = W/n!/2 > 


for large enough k, as desired. | 


Therefore growth rates of principal class grow at least quadratically in terms of 
the length of the avoided pattern, but the upper bound conjectured by Arratia is false 
(as we saw in Section 12.1.4). The fact that Arratia’s Conjecture failed for a layered 
permutation only lent more credence to the long-standing conjecture that among all 
permutations B of length k, the permutation that is easiest to avoid (thereby giving 
the greatest growth rate) is layered. In his survey for the 2013 British Combinatorial 
Conference, Steingrimsson [155] wrote that 


Although the evidence is strong in support of the conjecture that the most 
easily avoided pattern of any given length is a layered pattern, there is 
currently no general conjecture that fits all the known data about the 
particular layered patterns with the most avoiders. However, there are 
some ideas about what form the most avoided layered patterns ought to 
have, and specific conjectures that have not been shown to be false. 


The oldest of the specific conjectures Steingrimsson mentions appeared in Burstein’s 
1998 thesis [69]. Bona [52] made a competing conjecture in 2007 that the most easily 
avoided permutation of length k was 1@2162106---@21 for odd k and 16216 
21@---@21@ 1 for even k. 
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After these conjectures were made, Claesson, Jelinek, and Steingrimsson [76] 
established that for every layered permutation B of length k the growth rate of Av(B) 
is less than 4k? (we present their argument in Section 12.3.1). Bona [45] refined this 
approach to show that for his conjectured easiest-to-avoid permutations, the corre- 
sponding growth rates were at most 2.25k. Because of this, the function 


g(k) = max{gr(Av(B)) : |B| = k} 


was widely believed to grow quadratically. Thus it was quite a shock when Fox [88] 
showed in 2014 that g(k) grows faster than any polynomial. We prove a specialization 
of Fox’s Theorem in Section 12.2.5. 

Before moving on, because this section concerns estimates we briefly define the 
pieces of big-oh notation we use. Let f(n) and g(n) be positive functions. These 
functions are asymptotic, written f ~ g, if f(n)/g(n) > 1 asn > ~. We write f = 
O(g) if there is a constant C such that f(n) < Cg(n) for all sufficiently large n. 
We similarly write f = Q(g) if there is a constant c such that f(n) > cg(n) for all 
sufficiently large n. Finally, we write f = O(g) if both f = O(g) and f = Q(g). 


12.2.1 Matrices and the interval minor order 


In the context of principal classes, the most general results on growth rates have come 
from expanding the vista to the context of (zero/one) matrices. The study of patterns 
within matrices dates back to at least 1951, when Zarankiewicz [167] posed what 
is now known as the Zarankiewicz Problem. While his problem is often phrased in 
terms of bipartite graphs, what he actually asked was 


Soit Ry, ol n > 3, un réseau plan formé de n? points rangés en n lignes 
et n colonnes. Trouver le plus petit nombre naturel k2(n) tel que tout 
sous-ensemble de R,, formé de k(n) points contienne 4 points situés 
simultanément dans 2 lignes et dans 2 colonnes de ce réseau. 


Translated, the Zarankiewicz Problem asks for the minimum number of ones such 
that any way those ones are arranged in an n x n matrix there will be a 2 x 2 submatrix 
of all ones. 

Let us introduce some notation to discuss this and more general problems. The 
weight of the matrix M is the number of ones it contains, and we denote this quantity 
by wt(M). We also say that the matrix P is a submatrix of M if P can be obtained 
from M by deleting rows, columns, and ones (in this last case, changing them to 
zeros). A generalization of Zarankiewicz’s problem is to determine the asymptotics 
of the function 


ex(n; P) = max{wt(M) : M is ann x n matrix avoiding P as a submatrix}, 


and the quantity Zarankiewicz asked about was ex(n;J2) + 1, where we denote by Jx 
the k x k all-one matrix. 

Shortly after Zarankiewicz posed his problem, Kévari, Sd6s, and Turan [120] 
showed that the answer is asymptotic to n>/?, We include a short proof below. The 
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upper bound is from the original proof while the lower bound was presented by 
Reiman [143] in 1958. 


Theorem 12.2.2 The function ex(n;J) is asymptotic to n3/?. 


Proof. We begin with the upper bound. Take M to be a J2-avoiding n x n matrix of 
weight ex(n; Jz). For each i, let d; denote the number of ones in row i. Call two ones 
in the same row a couple, so row i contains (4) couples. Clearly M can have at most 
Gy total couples because the same two columns cannot participate as a couple in two 


different rows so . 
n d; 
> . 
()=¥ (3) 


Next we use the Cauchy-Schwarz inequality to conclude that Yd? > (Yj) /n. By 
noting that wt(M) = )dj, we obtain 


Pips ¥ dy gs we: 


showing that ex(n; Jz) does not grow asymptotically faster than nl? 

For the lower bound, let g be a prime power and suppose that n = q*+q+1. It 
is known that there is a finite projective plane of order g, with 1 points and n lines. 
Label the points p,,...,py and the lines ¢;,...,¢, and define the incidence matrix 
M by setting its (i,j) entry equal to one if and only if the point p; lies on the line 
¢;. Clearly M does not contain Jy, because any two points determine a unique line. 
Moreover, a basic counting argument shows that in a projective plane of order g every 
point lies on g+ 1 lines. Thus we have 


wt(M) = (q+1)(¢+q4+1) en, 
The fact that ex(n;J2) ~ n°/? follows from this and basic results about the distribution 
of primes. | 


As Kévari, Sd6s, and Turan [120] noted in their original paper, the upper bound 
in Theorem 12.2.2 can be generalized to show that ex(n;J,) = O(n?—!/) for all k 
and they conjectured that this is the true order of magnitude, i.e., that ex(n;J,) = 
@(n2-!/ k) While this conjecture is widely believed to be true, it remains open. The 
best general lower bound shows that ex(n;J,) = Q(n?-2/(K+)), 

Almost 40 years passed before the function ex(n; P) was investigated for matrices 
other than J;, in two papers published around the same time and, oddly enough, 
concerning the same forbidden pattern (up to symmetry). Bienstock and Gyori [38] 


established that 
.f( 11 = 
ex (ns ( i i )) = O(nlogn). 


(It is common practice in this field to suppress zeros.) Along with giving an indepen- 
dent proof of this result, Fiiredi [91] also presented a construction (set M(i,j) = 1 
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if and only if j —i is a power of 2) showing that this is the correct rate of growth. 
Both papers were motivated by problems in discrete geometry: Bienstock and Gyéri 
were investigating the complexity of an algorithm for computing obstacle-avoiding 
rectilinear paths in the plane, while Fiiredi used this extremal function to bound the 
number of unit distances between the vertices of a convex polygon. 

The first systematic investigation of these extremal functions was performed by 
Fiiredi and Hajnal [92] in 1992. They found the extremal functions (up to constant 
factors) for several matrices. Some of these extremal functions are quite exotic; for 
example, using the work of Hart and Sharir [100] Fiiredi and Hajnal showed that 


ex(m(' , 1 ,))=@(nain) 


where (1) is the inverse Ackermann function. At the end of their article, they asked 
is it true that the complexity of all permutation configurations are linear? 


The statement that ex(n; Mg) is linear in n for every permutation B became known as 
the Fiiredi—Hajnal Conjecture; here Mg denotes the permutation matrix of 8, which 
is defined by Mg (i, j) = 1 if and only if B (i) = j. Indeed, at the time Marcus and Tar- 
dos proved the Fiiredi—Hajnal Conjecture, they did not know about the Stanley—Wilf 
Conjecture, nor that Klazar had shown that the Fiiredi—Hajnal Conjecture implied the 
Stanley—Wilf Conjecture (we prove this in the next subsection). 

Despite the rich history of these submatrix problems, later advances—especially 
those of Fox [88]—show that the permutation containment order is much more 
closely related to a different order on matrices, the interval minor order. In retro- 
spect, this order has featured in the proofs of all of the main results presented in this 
section, before it had ever been formally defined. Such a formal definition requires 
a few preliminaries. The contraction of two adjacent rows of a matrix is obtained 
by replacing those two rows by a single row that has a one in a column if either of 
the original rows had a one in that column. We define the contraction of a column 
analogously. We further say that P is a contraction of M if P can be obtained from 
M by a sequence of contractions of adjacent rows or columns. 

To give an alternative definition of contractions, given an n x n matrix M and 
intervals X,Y C [1,n], we denote by M(X x Y) the block of M consisting of those 
entries M(i, 7) with (i,j) © X x Y = {(x,y) : x © X,y © Y}. The contraction P of 
M can be defined by a pair of sequences 1 = c) <--- < G41; =n-+1 (the column 
divisions) and 1 =r; <--- <7,41; =n+1 (the row divisions) where we set 


PE a= 1 if the block M([cj,ci+1) x [rj,rj+1)) contains a one and 
J) 0 otherwise. 


(We continue to use Cartesian coordinates so that the permutation matrix of 7 looks 
like its plot.) 

Finally, the matrix P is an interval minor of M if P is a submatrix of a con- 
traction of M. In other words, the interval minor order allows for the contraction of 
adjacent rows or columns, and the deletion of ones. As we will see, at the coarsest 
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Figure 12.15 
A 9 x 9 permutation matrix containing J3 as an interval minor. 


level, questions about growth rates of principal permutation classes are intimately 
connected with the following question: 


How much can ann x n matrix weigh if does not contain J; as an interval 
minor? 


We present two answers, in Sections 12.2.3 (an upper bound) and 12.2.5 (a lower 
bound). The connection between this question and growth rates is explored in Sec- 
tions 12.2.2 and 12.2.4. We begin with two elementary observations. 


Observation 12.2.3 [f the matrix M contains the permutation matrix Mg as an in- 
terval minor then it also contains Mg as a submatrix. 


Due to this fact, when we say that a matrix contains a permutation matrix we 
needn’t specify which order we are considering. Indeed, permutation matrices are 
essentially characterized by the condition in Observation 12.2.3—it is an easy exer- 
cise to show that if P is such that every matrix containing P as an interval minor also 
contains P as a submatrix then, modulo all-zero rows and columns, P is a permutation 
matrix. 

Observation 12.2.3 implies that if a matrix contains J; as an interval minor then 
it contains every k x k permutation matrix. Thus to get an upper bound on gr(Av(B)) 
we can consider the set of matrices that avoid J\g) as an interval minor. In particular, 
the Fiiredi—Hajnal Conjecture will follow if we can establish that, for every k, there 
is a constant cz such that wt(M) < cyn for every n x n matrix that does not contain Jy 
as an interval minor. 

The link between avoiding J; and lower bounds on growth rates comes via the 
following fact, whose truth is demonstrated by Figure 12.15. 


Observation 12.2.4 For every nonnegative integer k, there is a permutation B of 
length k? whose permutation matrix contains J, as an interval minor. 


Therefore if many permutation matrices avoid J, as an interval minor, they also 
avoid Mg for some permutation B of length k?. 


Permutation Classes T77 


Figure 12.16 
The interval minor M/? formed in the proof of Klazar’s Theorem 12.2.5. 


12.2.2 The number of light matrices 


The main result of this subsection provides the link between the extremal Fiiredi— 
Hajnal Conjecture and the enumerative Stanley—Wilf Conjecture. We state it in a 
slightly different way than it was originally presented. Let us define the density of 
the matrix M as 


t(P 
6(M) = max { 2) : Pis an m xX m interval minor omy. 
m 


Note that this definition measures the maximum average number of ones per row 
over all square interval minors of M. In particular, the density of a matrix can be 
greater than 1; for example, the permutation matrix shown on the left of Figure 12.15 
has density at least 3, because it contains J3 as a minor. 

We seek to bound the number of matrices of small density. 


Theorem 12.2.5 (Klazar [114]) Let d > 0 be a constant. There are fewer than 1524" 
matrices of size n x n and density at most d. 


Proof. We prove the theorem by induction on n. As the base case n = 1 is trivial, 
we may assume that n > 2 and that the claim holds for all lesser values of n. 

Let M be ann x n matrix of density at most d and denote by M /? the interval mi- 
nor of M formed by using the row and column divisions {1,3,5,...,2 +1}, without 
deleting any ones. Thus M /2 is formed by contracting 2 x 2 blocks of M (or possibly 
smaller blocks on the top and right), as shown in Figure 12.16. 

By the definition of density, wt(M/*) < d[n/2]. Now we ask how many n xn 
matrices could condense to a given [n/2] x [n/2] matrix M/*. Each nonzero entry 
of M/? can come from at most 15 possible blocks of M (all blocks except for the 
all-zero block), while the zero entries of M/* can only come from an all-zero block 
of M. Since M’? has at most d[n/2] nonzero entries, the number of n x n matrices 
that could condense to it is at most 154!”/?1, 

Therefore, the number of n x n matrices of density at most d is at most 1541n/2] 
times the number of possibilities for M/?, which by induction is less than 1524[n/2| 
In other words, the number of 1 x n matrices of density at most d is less than 


154/"/21 ‘ 1524/”/21 = 1534/”/21 < 1524" 


for n > 2, completing the proof of the theorem. | 
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This result shows that the Fiiredi-Hajnal Conjecture implies the Stanley—Wilf 
Conjecture as follows: if the Fiiredi—-Hajnal Conjecture is true, then for every per- 
mutation B there is a constant cg such that ex(n;Mg) < cgn for all n. Therefore 
every Mgp-avoiding matrix has density at most CB. Klazar’s Theorem 12.2.5 shows 
that there are at most (157¢8 )” such matrices, so there are only exponentially many 
such permutation matrices, so gr(Av(B)) is finite. 

As we have presented it, the bound in Klazar’s Theorem 12.2.5 is essentially best 
possible. To see this, let M be any n x n matrix with dn ones and density d. Then M 
itself contains 24” = (27)” submatrices of size n x n and density at most d. Thus to 
improve on the link between bounds in the Ftiredi—Hajnal Conjecture and those in the 
Stanley—Wilf Conjecture, one must restrict to permutation matrices. Via some quite 
delicate bounds, Cibulka [75] showed that the number of n x n permutation matrices 
of density at most d is at most (2.88d*)". Fox [88] gives a much simpler proof that 
the nth root of this number is O(d?). 


12.2.3. Matrices avoiding J; are light 


We now present Marcus and Tardos’ proof of the Fiiredi-Hajnal Conjecture. The 
general approach is again to consider a specific contraction. 


Theorem 12.2.6 (Marcus and Tardos [129]) Ifthe n xn matrix M does not contain 


Jj, as an interval minor then 
k2 
5(M) <ut(). 


Proof. Let f,(n) denote the maximum weight of an n xn matrix that avoids J; 
as an interval minor. We apply induction on n, noting that the n = 1 case is trivial. 
Let M be ann Xn matrix of weight f,(n) not containing J, as an interval minor. 
We form the interval minor M/4 by choosing our row and column divisions to be 
{1,q¢+1,2q+1,...n+1}; here q is a parameter to be determined at the end of the 
proof. Thus we are contracting M into q x qg blocks (possibly including smaller blocks 
on the top and right). Of course, M /4 also does not contain J; as an interval minor. 

The cleverest part of the proof is the following definition. A g x q block of M 
is called tall if it contains ones in at least k different rows and/or wide if it contains 
ones in at least k different columns. We begin by bounding the number of tall and/or 
wide blocks of M. 

Suppose that more than (k — 1) (7) tall blocks of M correspond to entries in the 
same row of M/4. We contract each such block into a single column. This gives an 
interval minor of M containing g rows and (k— 1) ( columns in which every column 
contains at least k ones. By deleting extraneous ones as needed, we may assume that 
each column has precisely k ones. However, that leaves us with only (4) choices of 
columns for this interval minor, and thus (under our assumption about the number of 
tall blocks in this row) at least one column must occur at least k times, showing that 
M contains J; as an interval minor, a contradiction. 
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By symmetry, at most (k — 1) (2) entries of each column of M/4 correspond to 
wide blocks in M. As M/4 has [n/q] rows and columns, we have the bound 


# of tall and/or wide entries of M/4 <2 =] (k—1) (7) ; 
q 


which is linear in n. For these entries of M/% we use the trivial bound that they 
correspond to blocks with at most g* nonzero entries. The remaining entries of M /a, 
corresponding to blocks that are neither tall nor wide, can have at most (k — 1)? 
nonzero entries. Putting these bounds together, we obtain by induction that 


wm so o(0)s4-74(F). 


In order to get a sense of the solution to this recurrence, we iterate it approximately: 


fe) S$ 2ante-(F) 4-1 ("), 
< 2an(a—1)(4) 46-1)? (ne—1)(2) 40-1 (4), 
< 2an(e—1)(4) +2ntk—18(4) 4220 —1)9(Z) 4 


We can further approximate this quantity by viewing it as a geometric series with 


ratio (k—1)?/q: 
fuln) $ 2an(k—1)(4) > i, 


i=0 q 


This series has a finite sum so long as we choose g > (k — 1)”, in which case we get 
a linear upper bound on wt(M), completing the proof. In particular, setting g = k*, 


we see that ; 
y (k-1)?\'_ ae 
= ke 2-1 , 


and the bound becomes ; 
k 
fu(n) S 2k" ( k ) Nn. 


While we presented only estimates above in order to motivate the choice of g, with 
this bound on f;,(1) now discovered it would only take a simple inductive argument 
to give a rigorous proof. i 


We have proved quite a bit more than the Stanley—Wilf Conjecture. Every proper 
downset of matrices in the interval minor order must avoid some matrix J;. Thus 
every proper downset of matrices in this order has bounded density by Marcus and 
Tardos’ Theorem 12.2.6 and thus grows at most exponentially by Klazar’s Theo- 
rem 12.2.5. Proper permutation classes are merely subsets of these downsets. 
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12.2.4 Dense matrices contain many permutations 


Having established upper bounds on growth rates, we now turn our attention to lower 
bounds. In this subsection we prove a simplified version of a result of Cibulka [75], 
showing that dense matrices contain many permutation matrices. In the following 
subsection, we use this result to show that growth rates of principal classes can be 
very large. Our first result in this direction, below, gives a lower bound for how many 
permutation matrices a dense matrix with a certain amount of regularity must contain. 


Proposition 12.2.7 (Cibulka [75]) Suppose that the matrix M has n columns and 
m >n rows, and that each column contains at least n ones. Then M contains at least 
nt/("") permutation matrices of size n x n. 


Proof. — Proceeding from left to right, we can find at least n! copies of n x n permu- 
tation matrices in M by choosing a nonzero entry in a new row from each column. 
Of course, some permutation matrices may be contained in M many times. However, 
M cannot contain more than (”") copies of a given n x n permutation matrix because 
that is the number of ways to select the rows from which the nonzero entries are then 
chosen. a 


The bound we actually use follows from Proposition 12.2.7 via Stirling’s For- 


mula: 
n! n! (n!)? n2\" 
= = > 
(7) ~ m"/n! m= ~ \ em 


n 


The main result of this subsection establishes the link between the weight of a 
matrix and the number of permutation matrices contained in it. Note that the conclu- 
sion of this theorem is only interesting for large densities 6(M). 


Theorem 12.2.8 (based on Cibulka [75, Theorem 6]) For every n x n matrix M, 
there is some value of m > 1 such that M contains ¥" or more m x m permutation 
matrices, where y = 5(M)!/9/16. 


Proof. The argument resembles that used to prove Marcus and Tardos’ Theo- 
rem 12.2.6, though with an additional clever idea and different parameters. In par- 
ticular, instead of being given k (formerly the size of the forbidden interval minor), 
here we choose k to be the smallest perfect square such that k > e+y”. Clearly we can 
satisfy this requirement for some k at most 2e*/. 

Define f;(n) to be the greatest possible weight of an n x n matrix that does not 
contain ¥” or more m x m permutation matrices for any value of m. The theorem is 
equivalent to the fact that 

fy(n) < (16y)?n, (12.1) 
which we establish by induction on n. If n < (167)?, then (12.1) holds trivially. Thus 
we may assume that n > (167)? and take M to be an n xn matrix of weight f,(1) 
that does not contain y” or more m x m permutation matrices for any value of m. 

Again we begin by forming the interval minor M/4 by choosing our row and 
column divisions to be {1,g+1,2qg+1,...n+1}, although this time we set g = 4k? 
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instead of k?. We have by induction that wt(M/“) < (16y)°[n/q]. The definitions 
of tall and wide are unchanged: a g x q block of M is tall (respectively, wide) if it 
contains ones in at least k different rows (respectively, columns). 

The next step is to bound the number of tall/wide blocks of M, but this time we 
obtain a different bound because of our assumption that M contains few permutation 
matrices. We bound the number of tall blocks that correspond to a single row of M /4, 
since the bound for wide blocks in a column will follow by symmetry. 

Given a set of tall blocks corresponding to entries in the same row of M /4, we 
group them into contiguous sets of k tall blocks apiece. Our bound comes from con- 
sidering two different types of groups. First, if the ones in all of the blocks of a group 
of tall blocks lie in only k3/? rows, we call the group concentrated. In this case, 
we form a minor by removing the all-zero rows of the block (this can be viewed as 
a contraction) and then contracting each block of the group to a single column. In 
doing so we obtain a matrix with k columns and at most k3/2 rows in which every 
column contains at least k ones (because the original blocks were tall). Applying 
Proposition 12.2.7 to this interval minor shows that it (and thus also M) contains at 


least : 

kd! > ( 2 )-(#) o 

(e?) = \ 6243/2] ~ \ @ = 

k 
k x k permutation matrices. Thus M cannot contain a concentrated group of tall 
blocks. 

It remains to consider groups of tall blocks that are not concentrated. If we have 

a nonconcentrated group of tall blocks then these blocks can all be contracted into 
a single column with at least k3/2 ones. Thus if M contains k°/* nonconcentrated 
groups then by contracting each group into a single column we obtain a matrix with 
k3/2 columns and q = 2k? rows in which each column contains at least k3/2 ones. 
Applying Proposition 12.2.7 to this interval minor shows that it contains at least 


3/2 3/2 
(ale ee Ae i Se ep \* er 
(2%) = \ 26242 ~ \ 2¢2 = he f 


K3/2 x 53/2 permutation matrices. Therefore M cannot contain k3/? nonconcentrated 
groups within the same row of blocks. 

We now count the ones in M. A given row of M /¢ must correspond to fewer than 
k5/2 tall blocks because when arranged in groups of k tall blocks apiece, no group 
can be concentrated, and M cannot contain k°/? nonconcentrated groups. Using the 
trivial bound that a q x q block can contain at most g* ones, we see that the combined 
weight of an entire row of tall blocks is at most g?k>/*. As there are [n/q] such rows 
of blocks and also [n/q] columns of blocks that are subject to analogous constraints, 
the total weight of all tall and/or wide blocks in M is less than 2[n/q]q?k?/?. 

Next we must bound the weight of the blocks that are neither tall nor wide. Each 
such block has weight less than k*. Moreover, the number of such blocks is at most 
wt(M/7) < (16y)?[n/q]. Combining our bounds (and remembering that q = 4k? 
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while k < 2e*y*) shows that 


wt(M) < 2 H Gk? + (167) |] R, 
q q 
97,2 
bag to OO )m 


Ce 
(190 + a n. 


It can be now be checked that 


9 
1662/2 < 16-29/2e!8y9 < vey . 


establishing (12.1) and completing the proof. | 


The clever new idea in the proof of Theorem 12.2.8 is that of concentration, 
which allows one to handle tall/wide blocks in either of two different ways. In 
Cibulka’s original proof he also refined the notion of tall/wide blocks, distinguishing 
between tall, very tall, and ultratall blocks. With these additional considerations, he 
was able to lower the 9 in the statement of the theorem to 4.5. 


12.2.5 Dense matrices avoiding J; 


Our final result of this section shows that the formerly widely held belief that 
gr(Av(B)) grows quadratically in |B| is false. This result, proved by Fox [88], is 
established by means of an elegant construction. 

Throughout this section we are concerned with a matrix M of size n x n where n 
is a power of 2, say n = 2”. We divide the interval [1,2”] into a number of dyadic in- 
tervals. The interval [1,2’] is itself dyadic. Its two halves, [1,2’—'] and [2’~! + 1,2"] 
are also dyadic. In turn, the halves of each of those intervals are dyadic, and this 
process continues until we reach the singletons, which are all dyadic. More formally, 
a dyadic interval is any interval of the form [a2 + 1,(a+1)2°] for nonnegative in- 
tegers a and b. A dyadic rectangle is then a product of two dyadic intervals (in the 
same sense as the products we used to define contraction). 

The number of dyadic intervals in [1,2”] is 


yi 2? =2"1-1=2n-1, 
b=0 


and every element of [1,7] lies in precisely r+ 1 such dyadic intervals (one of each 
size). The final observation we need to make about dyadic intervals is their most im- 
portant property, at least from the viewpoint of our upcoming construction. Suppose 
that J; and Jy are subintervals of [{1,n], and for i € {1,2} let D; denote the smallest 
dyadic interval containing J;. If D; = Do, then both J; and J; must contain elements 
of both halves of this dyadic interval. Therefore we can conclude that the smallest 
dyadic intervals containing two disjoint subintervals of [1,n] are distinct. 
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We are now ready to describe the construction of M. Order the dyadic subinter- 
vals of [1,n] as D1,...,D2n—1 arbitrarily and let A be a (2n — 1) x (2n — 1) auxiliary 
matrix. We build M from A by setting 


ice 1 if A(k,¢) = 1 for every pair k, @ with (i, 7) € Dy x Dp, 
J) 0 otherwise. 


The following result illustrates the usefulness of this construction. 


Proposition 12.2.9 (Fox [88]) Suppose that A and M are related as above. If A 
avoids J, as a submatrix, then M avoids J, as an interval minor. 


Proof. Suppose that M contains J; as an interval minor. Thus there are row and 
column divisions | =7) <---<7gyy =n+1l and l=c) <-:-<cpg,, =n+1 such 
that for every i and j, the block M((ci,ci+1) x [rj,rj+1)) contains a nonzero entry. 
Now choose indices ¢),...,¢€; such that for every i, the smallest dyadic interval 
containing [c;,cj41) is Dz, and similarly choose indices 7 ,...,7% such that for every 
j, the smallest dyadic interval containing [r;,rj+1) is D;,. Our observation above 
shows that because the intervals we are considering are disjoint, these are sets of 
distinct indices. Thus it follows by the construction of M that A(¢;,7;) = 1 for all 
indices i and j, i.e., that A contains J; as a submatrix, completing the proof. a 


As one might expect for an extremal result, the proof of Fox’s Theorem is proba- 
bilistic. In order to find a dense matrix M avoiding J; as an interval minor we choose 
the entries of the auxiliary matrix A uniformly at random and then (essentially) con- 
struct M as above. While this part of the proof is conceptually straight-forward, the 
reader may find the requisite inequalities a bit laborious. 


Theorem 12.2.10 (Fox [88]) For all sufficiently large positive integers k such that 
Vk/8 is an integer, there exists a matrix of density at least 2V4/16 that avoids JE as 
an interval minor. 


Proof. Suppose that r = Vk/ 8 is an integer and set n = 2” so that every integer in 
the interval [1,7] lies in precisely r + 1 dyadic intervals. The proof uses a parameter 
q, which is a probability that will be determined near the end and will satisfy 1/2 > 
q>4r/k. 

We begin by building the auxiliary matrix A of size (2n— 1) x (2n— 1). Choose 
each entry of A uniformly at random so that it is 1 with probability 1 — gq. Let X denote 
the random variable counting the number of copies of J; occurring as a submatrix in 


A. Clearly 
a= (""') aa". 


The standard bound | — x < e~* shows that (1 — qy < e~ 4°. For the binomial co- 


efficient, we have 
es ) 2 (nye ny > 2 
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where the second inequality follows because k > 4. Combining these bounds yields 


U[X] < nk eae 


The right-hand side of this inequality is maximized when gq is as small as possible. 
AS 


< 4r  Alogyn a 4Inn 
oa nn k 
we see that 


= 2k ,—4kInn __ )—2k 
H[X] < n““e Sn, 


In particular, 


Pr[A contains J; as a submatrix] < E[X] <n**, 


so almost all such matrices avoid J; as a submatrix. 

Now construct M as in Proposition 12.2.9. The probability that an entry of M is 
equal to | is thus (1 — qyrty because every entry lies in (r + 1)* dyadic rectangles. 
It is worth remarking that the entries of M are highly correlated (when viewed as n? 
random variables). In particular, the probability that M is the zero matrix is at least g, 
because one of the entries of A corresponds to the dyadic rectangle [1,n] x [1,7], and 
thus must be | if M is to have any nonzero entries. Of course, this correlation does 
not prevent us from appealing to linearity of expectation, which shows that 


e[we(M)] = n?(1—g) "4, 


However, we must guarantee that M avoids J; as an interval minor, and this con- 
struction does not provide such a guarantee (though it makes it exceedingly likely). 
Therefore we take M as above if it avoids J; as an interval minor (1.e., if A avoids 
J, as a submatrix), and otherwise set M equal to the zero matrix. From our previous 
computation of E[X], we see that 


i[wt(M)] > n(1— gyre —n?-Pr[A contains J; as a submatrix], 
_ n2(1 = qyetly pp 


> wis qyrry —1. 


bi 


Thus there is an n x n matrix M avoiding J; as an interval minor with weight at least 
this expectation. 

To finish the proof we need to choose g and then manipulate the inequalities to 
show that the density of M is at least 2ve/ 16 which is equivalent to showing that 
wt(M) > n°/*, For the rest of the proof we assume that k > 487 = 2308 and set 
q=1/ Vk. As promised at the beginning of the proof, 


V..» 1. ACES) 4r 
2° 8" rs 


We now claim that for this value of g and the particular matrix M we have selected, 


wt(M) > n2(1— 9)" —1 > 022734" —1 > 73/2. (12.2) 
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Of these three inequalities, the first follows from our choice of M and the sec- 
ond requires the most work. Canceling the 1 and the n*, we want to show that 
(1- q)rt? > n>", Taking logarithms of both sides, this is equivalent to 


(r+1)*log(1—q) > —(3log2)qr* (12.3) 


For q < 1/2, we have the bound log(1— qg) > —3q/2 so the left-hand side of (12.3) 
can be bounded by 


(r+1)*log(1 —q)> (r+). 


For r > 6 (which we have because k > 487), 
3 
=r 1)? > —(3log2)r? (= —2.0877) , 


verifying (12.3) and thus the second inequality of (12.2). 
All that remains is to show the final inequality of (12.2). Recall that n = 2”, so 


de 5] ay Oe a OP A: 
This quantity is greater than n3/2 = 2Vk/ 6p for n > 4, completing the proof. a 


It remains only to connect this result to growth rates of principal classes, which 
we do via our previous definition of 


g(k) = max{gr(Av(B)) : |B| =k}. 


Choose k large enough so that Theorem 12.2.10 holds. Recall by Observation 12.2.4 
that there are permutations of length k? that contain a J; minor. Let B be such a 
permutation and set ¢ = k?. Fox’s Theorem shows that there is a matrix M of density 
at least 2/""/!6 that avoids J, as a minor and thus also avoids the permutation matrix 
of B. Cibulka’s Theorem 12.2.8 then shows that there is a value of m such that M 
contains Y” or more m x m permutation matrices, where 


1/4 
ge 5(M)!/9 _ 2 /144 — 26'/4/144-4 
16 16 


Every permutation matrix contained in M also avoids the permutation matrix of B. 
From our observation in Section 12.1.1 about supermultiplicativity it follows that 
gr(Av(B)) > y, showing that 


g(k) = 20), 


Thus g(k) grows faster than every polynomial. (There is no commonly agreed 
upon term for functions that grow like 2" for 0 < a < 1. Because such functions 
grow much more slowly than 2”, they are often called subexponential when found 
as upper bounds for algorithmic problems. However, this term does not accurately 
convey how quickly such functions do grow, so terms such as stretched exponential 
and mildly exponential are also used.) 
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Figure 12.17 
A member of the infinite antichain commonly denoted U. 


12.3 Notions of structure 


We saw in the previous section that proper permutation classes have finite upper 
growth rates. However, none of those techniques are of any use if we want to know 
the precise growth rate of a class. Indeed, even if we only want to estimate growth 
rates, it is hard to imagine those techniques providing much insight. Thus it seems 
that to know more about growth rates of permutation classes we must take a detailed 
look at their structure. 

As we will see, there are a great many aspects of “structure” for permutation 
classes that have so far resisted unification. For a first aspect of structure, consider the 
permutation shown on the left of Figure 12.17. In the middle and right of this figure, 
two drawings of the permutation graph, G7, of this permutation are presented. This 
is the graph on the vertices {(i,2(i))} in which (i, 2(i)) and (j,a(j)) are adjacent if 
they form an inversion, i.e., i << j and (i) > 2(j). The drawing on the right of this 
figure shows that the permutation graph of this permutation is a split-end path, that 
is, it is constructed from a path by adding four vertices, two adjacent to each leaf. 
Clearly we could modify this construction to build an infinite set of permutations 
whose permutation graphs are all split-end paths. 

It is easy to see that if o < z then Gg is an induced subgraph of Gz, though 
the converse need not hold (for example, Gz and G,,-1 are isomorphic, but of course 
unless 7 is an involution, it is not contained in m'), Therefore, since no split-end 
path is an induced subgraph of another, it follows that the infinite set of permutations 
built this way is an infinite antichain (recall that an antichain is a set of pairwise in- 
comparable permutations). The same approach shows that the family of permutations 
alluded to in Figure 12.11 also forms infinite antichain; the graph of the permutation 
from that figure is shown below. 


7] es, 


A well-quasi-order (wqo) is a quasi-order (a binary and transitive, but not 
necessarily antisymmetric, binary relation) that contains neither an infinite strictly 
descending sequence nor an infinite antichain. Because containment is a partial or- 
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der on permutations, we use the term well-partially-ordered (wpo) to describe this 
property instead. Of course, permutation classes cannot contain infinite strictly de- 
creasing sequences, so wpo is synonymous with the absence of infinite antichains in 
this context. Well-quasi/partial-orders have been studied quite extensively in a vari- 
ety of contexts. Cherlin [74] surveys these investigations from a particularly general 
perspective. 

The infinite antichain we constructed above from split-end paths lies in the class 
Av(321). Atkinson, Murphy, and Ruskuc [23] showed that the only wpo principal 
classes are Av(1), Av(21), Av(231), and their symmetries. This supports our state- 
ment in Section 12.1.2 that despite their Wilf-equivalence, the classes Av(231) and 
Av(321) are very different. 

The following property of wpo classes can be tremendously useful. 


Proposition 12.3.1 The subclasses of a wpo permutation class satisfy the descend- 
ing chain condition, i.c., if @ is a wpo class, there does not exist an infinite sequence 
C=O 2€'DEC Q--- of permutation classes. 


Proof. | Suppose to the contrary that the wpo class @ were to contain an infinite 
strictly decreasing sequence of subclasses @ = @° D @! DG D---. Foreachi> 1, 
choose B; € | \ @'. The set of minimal elements of {B1,B2...} is an antichain 
and therefore finite, so there is an integer m such that {B), B2..., 8} contains these 
minimal elements. In particular, B,,,1 > B; for some 1 <i < m. However, we chose 
Bnvi € @"\ G"*!, and because B,,.; contains B;, it does not lie in @ and thus 
cannot lie in @”", a contradiction. | 


In their 1996 paper, Noonan and Zeilberger [134] conjectured that every finitely 
based permutation class has a D-finite generating function. Clearly the finite basis 
hypothesis is necessary—because there are infinite antichains of permutations, there 
are uncountably many permutation classes with distinct generating functions, but 
only countably many D-finite generating functions with rational coefficients. The 
status of the Noonan-Zeilberger Conjecture is still unresolved, though Zeilberger no 
longer believes it to be true (as witnessed by his quote about Av(1324) presented 
in Section 12.1.4). Moreover, the work of Conway and Guttman [78] suggests that 
Av(1324) has a non-D-finite generating function, giving us a concrete potential coun- 
terexample. 

One might also ask about the other direction. If a class has a “nice” generat- 
ing function, does it have a good deal of structure? Indeed, at the same confer- 
ence, Zeilberger asked for necessary and sufficient conditions for a class to have a 
rational/algebraic/D-finite/... generating function [81]. However, the work of Albert, 
Brignall, and Vatter [12] shows that this question is almost certainly intractable. 

To briefly sketch this argument, let @ be a proper permutation class, so Marcus 
and Tardos’ Theorem 12.2.6 shows that there is some constant y such that |@;,| < 7" 
for all n. By modifying the construction of the antichain in Figure 12.17, it can be 
shown that for every finite y, there is an infinite antichain A containing at least 6” 
permutations of each sufficiently long length n for some constant 6 > y. Moreover, 
A can be constructed in such a way that its downward closure, Sub(A), has a rational 
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generating function. Therefore if we start with the class @ U Sub(A) and remove 
|G,| elements of length n from A for every sufficiently long length n, we obtain a 
permutation class containing @ that has (up to the addition of a polynomial) the 
same rational generating function as Sub(A), proving the following. 


Theorem 12.3.2 (Albert, Brignall, and Vatter [12]) Every permutation class ex- 
cept for the class of all permutations is contained in a class with a rational gen- 
erating function. 


In light of Theorem 12.3.2, it is clearly hopeless to attempt to establish a struc- 
tural characterization of classes with rational (or algebraic, or D-finite, ...) generating 
functions. A similar issue arises in the definition of perfect graphs. In that context, 
we know that the chromatic number of a graph, 7(G), is at least its clique number, 
@(G) and we would like to ask which graphs achieve equality, but we’re faced with 
the problem that given any graph, its union with a sufficiently large complete graph 
will satisfy 7 = @. Thus we say that a graph is perfect if 7 = @ for the graph and all 
of its induced subgraphs. In the permutation class context, we parallel this by saying 
that a permutation class is strongly rational (respectively, strongly algebraic) if it 
and all of its subclasses have rational (respectively, algebraic) generating functions. 
Our counting argument from before now yields the following implication. 


Proposition 12.3.3 Every strongly algebraic permutation class is wpo. 


I have conjectured that this is actually the characterization of strongly algebraic 
permutation classes, though this conjecture has never before appeared in print. 


Conjecture 12.3.4 A permutation class is strongly algebraic if and only if it is wpo. 


As of yet, very little has been established about strongly algebraic classes, but 
there are some results on strongly rational classes. Using the substitution decom- 
position (the topic of Section 12.3.2), Albert and Atkinson [3] proved that every 
proper subclass of Av(231) has a rational generating function. More generally, Al- 
bert, Atkinson, and Vatter [8] showed that in a strongly rational class, the sum inde- 
composable permutations also have a rational generating function. Other properties 
of strongly rational classes are discussed in Section 12.4.2. 

In the rest of this section we present several more notions of structure and study 
their interactions. Most of the tools developed here are applied in the final section, 
where we ask about the set of all growth rates of permutation classes. 


12.3.1 Merging and splitting 


Here we consider a very coarse notion of structure. Given any two permutation 
classes @ and JY, their merge, @ © J, consists of those permutations whose entries 
can be partitioned into two subsequences, one order isomorphic to a permutation in 
€ and the order isomorphic to a permutation in F (usually thought of as coloring the 
entries of the permutation red or blue). For example, Proposition 12.1.4 shows that 


Av(321) = Av(21) © Av(21), 
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and this generalizes to any class of the form Av(k---21). What if we change one of 
the Av(21) classes to Av(12)? Then we obtain the class of permutations that can be 
expressed as the union of an increasing subsequence and a decreasing subsequence. 
This class was introduced by Stankova [150] who named it the class of skew-merged 
permutations and computed its basis: 


Av(21) © Av(12) = Av(2143, 3412). 


Atkinson [20] was the first to compute the (algebraic) generating function of this 
class. 

It should be noted that the merge operation preserves very few properties. The 
example of Av(321) shows that it does not preserve wpo or rational generating func- 
tions, while the example of Av(4321) shows that it does not preserve algebraic gener- 
ating functions. Moreover, it is relatively easy to construct examples of finitely based 
classes whose merge is not finitely based, so merge does not preserve finite bases 
(though Kézdy, Snevily, and Wang [110] proved that the merge of Av(12---k) with 
Av(€---21) always has a finite basis). However, we can get a bound on the growth 
rates of merges by the following result (which seems to have first appeared, though 
implicitly, in Albert [1]). 


Proposition 12.3.5 For any two permutation classes 6 and Y, 
(COD) < <( a(O)+ /a(D y) , 


Proof. Given a permutation of length k from @ and another permutation of length 


2 ; 
n—k from &%, there are () ways to merge them to form a permutation of length 
n: choose k positions to be occupied by the permutation from @, then choose the k 
values for this subsequence, and then the permutation is determined. Therefore 


n 2 n Z 
Keo mnlsd (7) raliaesl< (¥ (7) via a] aay 
k=0 k=0 


To avoid introducing epsilon, we sketch the proof from this point, though it is not 
difficult to make it formal. Suppose that |@,| ~ y” and |Y,| ~ 6”. Then (12.4) be- 


comes 3 
n n —\ 2n 
(COD AS|Y ({) Vike = (V7+v8) , 
k=0 
Taking nth roots gives the desired inequality. | 


Which classes can we obtain via merges? Let us say that the class @ is splittable 
if @ is contained in Y©E for proper subclasses J,E C @. Thus Av(k...21) is split- 
table for every k, as is the class of skew-merged permutations. But clearly the class 
Av(21) of increasing permutations is not splittable, as is (with a bit more thought) 
the class of layered permutations. In its fullest generality, this remains an open (and 
seemingly quite difficult) question. 
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Figure 12.18 
The final contradiction in the proof of Proposition 12.3.7. 


Question 12.3.6 Which permutation classes are splittable ? 


Question 12.3.6 remains open even for principal classes. The case of Av(f) 
where B is layered has been considered several times in the literature. Bona [52] 
presented the first splittability result for such classes, though not in this language. 
His result was then generalized by Claesson, Jelinek, and Steingrimsson [76], which 
was in turn generalized by Jelinek and Valtr to the following. 


Proposition 12.3.7 (Jelinek and Valtr [105]) For all nonempty permutations a, B, 
and Y, we have 
Av(a@dB Py) C Aviad B)OAvV(B ey). 


In particular, every principal class whose basis element is the sum of three (or more) 
nonempty permutations is splittable. 


Proof. Take 7 € Av(a@® PB @y). We seek to color the entries of 2 red and blue so 
that the red entries avoid @ @ B and the blue entries avoid B © y. We proceed from 
left to right, coloring the entry 7(i) red unless 


(B1) doing so would create a red copy of a & B, or 
(B2) there is already a blue entry below and to the left of z(i). 


If either of these conditions hold, we color (i) blue. 

The red entries of z avoid @@ B by definition, so it suffices to show that the blue 
entries avoid B @ y. Suppose otherwise, and let 2(k) denote the bottommost entry of 
a blue copy of B @y in a. If 2(k) were colored blue because of rule (B1), then 2 
would contain a copy of @ @ B ending at m(k), and thus this together with the blue 
copy of y shows that z would contain a @ B @y. As this leads to a contradiction, 
7(k) must have been colored blue because of rule (B2). 

Let 2(j) denote the leftmost blue entry below and to the left of 2(k). By this 
choice of 2(j), we know that it was colored blue because of rule (B1), and thus it 
forms the rightmost entry of an otherwise-red copy of @@ B in 2. We now seek to 
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establish the contradiction shown in Figure 12.18. Choose 2(i) to be the leftmost 
entry in the blue copy of B @ y which z(k) lies in. Clearly if the red copy of were 
to lie completely to the left of z(i) then z would contain a @ B @ y, so some entries 
of this copy of a must lie to the right of 2(i). However, this means that all of the 
entries of the mixed copy of B must lie to the right of z(i), and thus because (i) is 
blue, none of the red entries of this copy of B may lie above 7(i). Finally, the blue 
entry 7(j) must also lie below (i) because it lies below 2(k). However, this implies 
that the mixed copy of 6 must lie entirely below and to the left of the blue copy of y, 
contradicting our assumption that 2 avoids a6 B Gy. a 


While the work of Jelinek and Valtr [105] focuses on the abstract notions of 
splittability, the earlier work of Claesson, Jelinek, and Steingrimsson [76] was con- 
cerned with the applications of Propositions 12.3.5 and (their version of) 12.3.7 to 
growth rates of principal classes with layered basis elements. In particular, we see 
that Av(1324) C Av(132) © Av(213), and thus the growth rate of Av(1324) is at 
most 16. Bona [44] has since refined this approach to give an upper bound of 13.74. 

Our next result was promised in the beginning of Section 12.2. 


Theorem 12.3.8 (Claesson, Jelinek, and Steingrimsson [76]) For every layered 
permutation B of length k, the growth rate of Av(B) is less than 4k’. 


Proof. Given a positive integer 0, we denote by 6, the permutation ¢---21. We 
prove the stronger inequality that for a sequence ¢1,...,n of positive integers sum- 
ming to k, 


gr(Av(dp, @-+-@&;,,)) < (2k- £1 — Ln —m +1)? < 4k?. 


The proof is by induction on m. The m = 1, 2 cases follow from Corollary 12.1.13, 
sO We may assume that m > 3. By Propositions 12.3.5 and 12.3.7, 


[stl Av(5r, ©: 5y,)) <_ y/gr(Av(Sr, © 5) + \/sr(Av(6, @---@5,.)); 
(€) +4. —-1)4+ (2(k- 41) — 62 — ln — (m—1) +1), 
= 2k-L;—ln—m—, 


IA 


as desired. | 


We conclude this subsection by returning to a (very) special case of the split- 
tability question. What about principal classes for which the basis element is the 
sum of just two permutations, i.e., those of the form Av(a@ @ B)? Here we can apply 
Proposition 12.3.7 to see that 


Av(a@B) C AvV(a@1@B) C Av(aG1)OAv(1 OB). 


Thus so long as neither @ @ 1 nor 1 B is equal to ~@ @ B, such classes are splittable. 
By symmetry, this leaves only those classes of the form Av(1 @ 8). We have already 
remarked that Av(12) is not splittable. Jelinek and Valtr [105] showed that Av(132) 
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is also not splittable. However, via an intricate argument, they were able to prove that 
every class of the form Av(1 @ B) with |B| > 3 is splittable, which is the final piece 
needed to obtain the following result. 


Theorem 12.3.9 (Jelinek and Valtr [105]) For all sum (or skew) decomposable 
permutations B of length at least four, the class Av(B) is splittable. 


12.3.2 The substitution decomposition 


While the merge construction can provide somewhat reasonable upper bounds on 
growth rates on splittable classes, it sheds little light on the exact enumeration prob- 
lem. In this subsection, we investigate the substitution decomposition, which has 
proved very useful for computing generating functions of classes, especially when 
combined with the techniques of Section 12.4.2. It should be noted that, as with most 
of the structural notions discussed in this section, the substitution decomposition has 
been studied for a wide variety of combinatorial objects. For a slightly outdated sur- 
vey, we refer to MoOhring and Radermacher [132]. This concept dates back to a 1953 
talk of Fraissé [89], although its first significant application was in Gallai’s 1967 
paper [93] (see [94] for a translation). 

An interval in the permutation 2 is a set of contiguous indices J = [a,b] such that 
the set of values z(/) = {z(i) : i € I} is also contiguous (four intervals are indicated 
in the permutation on the left of Figure 12.19). Given a permutation o of length m 
and nonempty permutations (,...,Qm, the inflation of o by Q,...,Qn, denoted 
O[O,...,Qm], is the permutation of length |a|+---+ || obtained by replacing 
each entry o(i) by an interval that is order isomorphic to q; in such a way that the 
intervals themselves are order isomorphic to o. For example, the permutation shown 
in Figure 12.19 is 

2413/1, 132,321, 12] = 4 798 321 56. 


Every permutation of length n > 1 has trivial intervals of lengths 0, 1, and n; all 
other intervals are termed proper. We further say that the empty permutation and 
the permutation | are trivial. A nontrivial permutation is simple if it has no proper 
intervals. The shortest simple permutations are thus 12 and 21, there are no simple 
permutations of length three, and the simple permutations of length four are 2413 
and 3142. Simple permutations and inflations are linked by the following result. Its 
proof follows in a straight-forward manner once one establishes the fact that the 
intersection of two intervals is itself an interval. 


Proposition 12.3.10 (Albert and Atkinson [3]) Every nontrivial permutation 7 is 
an inflation of a unique simple permutation 0. Moreover, if ™ = 0[Q,...,Qm]| for 
a simple permutation o of length m > 4, then each Qj; is unique. If 7 is sum de- 
composable, then there is a unique sequence of sum indecomposable permutations 
Q,.--,Qm such that 7 = A ®-+: Pp. The same holds, mutatis mutandis, with sum 
replaced by skew sum. 
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The plot of the permutation 479832156 and its substitution decomposition tree. 
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By recursively decomposing the permutation 7 and its intervals as suggested by 
Proposition 12.3.10, we obtain a rooted tree called the substitution decomposition 
tree of z (an example is shown on the right of Figure 12.19). The substitution depth 
of 7 is the height of its substitution decomposition tree, so for example, the substi- 
tution depth of the permutation from Figure 12.19 is 3, while the substitution depth 
of every simple or monotone permutation is 1, and the substitution depth of every 
nonmonotone layered permutation is at most 2. 

The main result of Albert and Atkinson’s formative paper on the substitution 
decomposition is the following. 


Theorem 12.3.11 (Albert and Atkinson [3]) Every permutation class with only 
finitely many simple permutations is strongly algebraic (and thus in particular, wpo). 
Moreover, every permutation class with only finitely many simple permutations and 
bounded substitution depth is strongly rational. 


For example, Theorem 12.3.11 shows that the class of separable permutations is 
strongly algebraic, because the only simple permutations in this class are {1, 12,21}. 
However, there are (relatively speaking) few permutation classes with only finitely 
many simple permutations, so it may appear that the substitution decomposition is 
not as applicable as might be hoped. We remedy this when we present a generaliza- 
tion in Section 12.4.2 (Theorem 12.4.16). 

A different generalization is due to Brignall, Huczynska, and Vatter [65]. They 
gave a proof of Theorem 12.3.11 using “query-complete sets of properties” show- 
ing that the conclusion holds not just for the permutation class itself but for a great 
many subsets of it. For example, if a permutation class contains only finitely many 
simple permutations then the generating function for the alternating (up-down) per- 
mutations, or the even permutations, or the involutions in the class is also algebraic. 

Given two classes Y and Y, the inflation of Z by Y is defined as 


QU ={olm,...,&n] : © € Dy and %,...,O0m € V}. 


The class @ is said to be substitution closed if @|@] C @. The substitution closure 
(@) of aclass @ can be defined in any number of ways, for example as the smallest 
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substitution closed class containing @, or as the largest permutation class with the 
same set of simple permutations as @, or as 


(@) =CUE|E|US[E[@]]U-- 


We leave it to the reader to convince themselves that these are all equivalent defini- 
tions and then to establish the following. 


Proposition 12.3.12 A class is substitution closed if and only if all of its basis ele- 
ments are simple permutations. 


Atkinson, RuSkuc, and Smith [26] investigated bases of substitution closures of 
principal classes. In a great many cases these bases are infinite, but they were able to 
compute several bases in the finite cases. For example, they showed that 


(Av(321)) = Av(25314,35142,41352, 42513, 362514,531642) 


by bounding the length of potential basis elements of this class and then conducting 
an exhaustive search by computer. 

If @ C (Q) for a proper subclass Y ¢ @ then we say that @ is deflatable. We 
then have the following analogue of the splittability question. 


Question 12.3.13 Which permutation classes are deflatable ? 


An equivalent definition is that the class @ is deflatable if and only if one can 
find a permutation 2 € @ that is not contained in any simple permutation of @. For 
example, the class of separable permutations is clearly deflatable, as it is equal to 
({1,12,21}). It follows that Av(231), as a subclass of the separable permutations, 
is also deflatable. Neither of these classes is splittable, so there are classes that can 
be decomposed via inflations that can’t be decomposed via the merge operation. In 
the other direction, Av(321) is trivially splittable, but it can be shown that it is not 
deflatable. 

While using the substitution decomposition to enumerate deflatable classes has 
become a common technique (several applications of this approach are cited in Sec- 
tion 12.4.2), Question 12.3.13 has only recently attracted attention. Interestingly, the 
most comprehensive result on deflatability to date, quoted below, shows that most 
of the classes known to be splittable by Theorem 12.3.9 are not deflatable. Thus 
deflatability and splittability are in some sense good complements to each other. 


Theorem 12.3.14 (Albert, Atkinson, Homberger, and Pantone [6]) For all choices 
of nonempty permutations a, B, and y, the class Av(a® B @ 7) is not deflatable. 
Moreover, for |a|,|B| > 2, the class Av(a @ B) is not deflatable. 


They also showed that all classes of the form Av(1@ B) with |B| < 4 are not 
deflatable, but the class Av(1 @ 23541) is deflatable. 

Atkinson posed a much different conjecture about the simple permutations in a 
class at the conference Permutation Patterns 2007. Let Si,(@) denote the set of all 
simple permutations in @ of length n. 
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Conjecture 12.3.15 (Atkinson) The ratio |Si,(@)|/|G,| tends to 0 as n > © for 
every proper permutation class @. 


What happens if we let @ be the (nonproper) class of all permutations in Atkin- 
son’s Conjecture? It is relatively easy to show that the probability that a permutation 
of length n contains an interval of size 3 or greater tends to 0 as n + -~, so (asymp- 
totically) the only obstructions to simplicity are intervals of size 2. Let X denote the 
number of intervals of size 2 in a permutation of length n chosen uniformly at ran- 
dom. It is routine to compute that E[X] = 2. Moreover, in the 1940s, Kaplansky [108] 
and Wolfowitz [166] showed (independently) that X is asymptotically Poisson dis- 
tributed, and thus P[X = 0] = 1 /e” (a more modern proof using what is known as the 
Chen-Stein Method appears in Corteel, Louchard, and Pemantle [79], and it is not 
difficult to establish this result via Brun’s Sieve if one is so inclined). 

Therefore the number of simple permutations of length n is asymptotic to n! /e?. 
Albert, Atkinson, and Klazar [7] gave refined asymptotics for this quantity, showing 


that it is 
n! 1 4 2, Lb 1 
e n n(n—1) | nm} 


They also established that the sequence counting simple permutations of length n is 
not D-finite. 

Our next result can be used to verify that a given permutation class contains only 
finitely many simple permutations. 


Theorem 12.3.16 (Schmerl and Trotter [147]) Every simple permutation of length 
n contains a simple permutation of lengthn—1 orn—2. 


It should be noted that Schmerl and Trotter proved their theorem in the more 
general context of binary, irreflexive relational structures, of which permutations are 
a special case. A streamlined proof of Schmerl and Trotter’s theorem for the special 
case of permutations is given in Brignall and Vatter [67]. 

The simple permutations that do not contain a simple permutation with one fewer 
entry are quite rare. We say that an alternation is a permutation whose plot can be 
divided into two halves, by a single horizontal or vertical line, so that for every pair of 
entries from the same part there is an entry from the other part that separates them, 
i.e., there is an entry from the other part that lies either horizontally or vertically 
between them. A parallel alternation is an alternation in which these two sets of 
entries form monotone subsequences, either both increasing or both decreasing (the 
permutation on the left of Figure 12.20 is a parallel alternation). The statement of 
Schmerl and Trotter’s Theorem can be refined to say that every simple permutation 
that is not a parallel alternation contains a simple permutation with one fewer entry. 

A wedge alternation is an alternation in which the two halves of entries form 
monotone subsequences, one increasing and one decreasing. Wedge alternations can 
be made simple by the addition of a single entry in one of two positions, as shown 
in the second and third permutations of Figure 12.20. We call the resulting permu- 
tations wedge simple permutations. Brignall, Huczynska, and Vatter [64] proved a 
Ramsey-like result for simple permutations, which states that every “long enough” 
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Figure 12.20 
From left to right, a parallel alternation, two wedge simple permutations, and a proper 
pin sequence. 


simple permutation contains a large simple parallel alternation, a large wedge sim- 
ple permutation, or a large member of a third family of simple permutations, called 
proper pin sequences. This latter family (a member of which is shown on the far 
right of Figure 12.20) is defined inductively. 

An axes-parallel rectangle is any rectangle of the form X x Y for intervals X 
and Y. The rectangular hull of a set of points in the plane is defined as the smallest 
axes-parallel rectangle containing them. Given independent points {p1,...,p;} in 
the plane, a proper pin for these points is a point p that lies outside their rectangular 
hull and separates p; from {p1,...,pi—i1}. A proper pin sequence is then constructed 
by starting with independent points p; and p2, choosing p3 to be a proper pin for 
{p1,p2}, then choosing p4 to be a proper pin for {p1, p2, p3}, and so on. It can be 
shown that every proper pin sequence is either simple or can be made simple by the 
removal of a single point. We can now state the theorem. 


Theorem 12.3.17 (Brignall, Huczynska, and Vatter [64]) There is a function f(k) 
such that every simple permutation of length at least f(k) contains a simple per- 
mutation of length at least k that is either a parallel alternation, a wedge simple 
permutation, or a proper pin sequence. 


By considering each of the three types of simple permutations guaranteed by 
Theorem 12.3.17, this result implies that there is a function g(k) such that every 
simple permutation of length at least g(k) contains two simple permutations of length 
k that are either disjoint or nearly so (they might share a single entry). 

This corollary puts the results of Bona [47, 48] under the substitution decompo- 
sition umbrella. He showed that for any value of r, the number of permutations with 
at most r copies of 132 has an algebraic generating function. (Mansour and Vain- 
shtein [128] later described a way to compute these generating functions.) Clearly 
the class of 132-avoiding permutations contains only finitely many simple permu- 
tations (it is a subclass of the separable permutations). To put this another way, all 
simple permutations of length at least 4 contain 132. Therefore, by the above corol- 
lary to Theorem 12.3.17, all simple permutations of length at least g(4) contain 2 
copies of 132. More generally, the class of permutations with at most r copies of 
132 does not contain any simple permutations of length g”(4) or longer, and thus has 
an algebraic generating function by Theorem 12.3.11. (A simpler derivation of this 
implication of Theorem 12.3.17 is given in Vatter [160].) 
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As shown in Brignall, RusSkuc, and Vatter [66], Theorem 12.3.17 can be used to 
devise a procedure to determine whether a class (specified by a finite basis) contains 
infinitely many simple permutations. Bassino, Bouvel, Pierrot, and Rossin [33] have 
since given a much more practical algorithm for this decision problem. 

We conclude this subsection by returning to the splittability question. The fol- 
lowing is a special case of the results of Jelinek and Valtr [105]. 


Proposition 12.3.18 Substitution closed classes are not splittable. 


Proof. Let @ be a substitution closed class. If 12 ¢ @ then @ must be the class 
of all decreasing permutations, which is clearly not splittable, so we may assume 
that 12 € @. Now take two proper subclasses Y,E € @ and choose o € @ \ FY and 
t€ @\E. Because 12 € @, the permutation p = o ®t = 12[o,7] lies in @ but in 
neither J nor E. 

Now consider 7 = p[p,...,P]. By definition, 2 € @, but we claim that 7¢ JOE 
Indeed, if we tried to color the entries of 7 red and blue so that the red subpermutation 
lied in Y and the blue subpermutation lied in E, we would have to use both colors 
in each interval order isomorphic to p (because p ¢ UE), but then there would be 
monochromatic copy of p containing one entry per interval. | 


By Proposition 12.3.18 and our previous estimates on the number of simple per- 
mutations, we see that if B is chosen uniformly at random from all permutations of 
length k, there is (asymptotically) a 1/e* probability that it is simple, and thus that 
Av(f) is not splittable. As Jelinek and Valtr [105] note, it would be interesting to 
obtain a better understanding of the probability that Av(B) is not splittable. 


12.3.3 Atomicity 


To motivate the notion of atomicity, let us continue with the splittability question. If 
the class @ can be expressed as the union of two of its proper subclasses, then it is 
trivially contained in the merge of these two classes. We say that @ is atomic if it 
cannot be expressed as the union of two of its proper subclasses. Thus the splittability 
question is only interesting for atomic classes. 

Atomicity was first used in the context of permutations by Atkinson in his 1999 
paper “Restricted permutations” [21] (which anticipated many of the structural no- 
tions discussed in this section), where he showed that 


Av(321,2143) = Av(321,2143, 2413) UAv(321,2143,3142), 


and used this to enumerate the former class. In the wider context of relational 
structures, the notion has a much longer history, dating back to a 1954 article of 
Fraissé [90]. It is not difficult to show that the joint embedding property is a nec- 
essary and sufficient condition for the permutation class @ to be atomic; this condi- 
tion states that for all o, tT € @, there is a a € @ containing both o and T. Fraissé 
established another necessary and sufficient condition for atomicity, which we de- 
scribe only in the permutation context (Fraissé proved his results in the context of 
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arbitrary relational structures). Given two linearly ordered sets A and B and a bijec- 
tion f : A — B, every finite subset {a; <--- < a,} CA maps to a finite sequence 
f(a1),---,f(an) € B that is order isomorphic to a unique permutation. We say that 
this permutation is order isomorphic to f({a1,...,a@n}) and call the set of all per- 
mutations that are order isomorphic to f(X) for finite subsets X C A the age of f. 
While this class is typically denoted Age(f : A > B), given our previous definitions 
we choose instead to refer to it as Sub(f : A > B). 


Theorem 12.3.19 (Fraissé [90]; see also Hodges [101, Section 7.1]) The following 
are equivalent for a permutation class ©: 


(1) @ is atomic, 
(2) © satisfies the joint embedding property, and 
(3) @ =Sub(f : A > B) for a bijection f and countable linear orders A and B. 


Proof. We first show that (1) and (2) are equivalent. Suppose to the contrary that @ 
satisfies the joint embedding property but @ C WUE for proper subclasses J,E C @. 
Clearly we may assume that neither Y nor E is a subset of the other, so there are 
permutations o € Y\E and t € E\ J. By the joint embedding property there is 
some 2 € @ containing both of these permutations, but 2 cannot lie in either J or E, 
a contradiction. Next suppose that @ does not satisfy the joint embedding property, 
so there are permutations o,t € @ such that no permutation in @ contains both. 
Therefore every permutation in @ avoids either o or T, so we see that @ is contained 
in the union of its proper subclasses @ M Av(o) and @ M Av(t). 

Next we show that (2) and (3) are equivalent. Suppose that @ = Sub(f : A > B) 
and take o,t € @. Thus o is order isomorphic to f(Ag) for a subset Ag CA and T 
is order isomorphic to f(Az) for a subset A; C A, so the permutation that is order 
isomorphic to f(Ag UA) contains both o and t. Next suppose that @ satisfies the 
joint embedding property and list the elements of @ as 0),02,.... Define 7 to be 
the empty permutation, and for i > 1, choose a permutation in @ that contains both 
0; and 7;_ to be 7;. Thus every element of @ is contained in some 7; (and thus also 
all 2; for j > i). Clearly we may choose Ao, Bo, and fo so that 7 is order isomorphic 
to fo(Ao). Now for i > 1, we construct A;, B;, and f; so that Aj; > Aj_1, Bj D Bi-1, 
fi(Aj) is order isomorphic to 7;, and such that f;(a) = fi-1(a) for all a € Aj_1. The 
desired bijection is then f = lim;_,.. fj, with A = JA; and B = UB;. | 


Recall that every principal permutation class is either sum or skew closed (Ob- 
servation 12.1.2), so all principal classes satisfy the joint embedding property and 
are trivially atomic. Despite this, the notion of atomicity has proved to be quite use- 
ful in the study of permutation classes. In particular, we apply the following two 
propositions in our final subsection. 


Proposition 12.3.20 Every wpo permutation class can be expressed as a finite union 
of atomic classes. 
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Proof. | Consider the binary tree whose root is the wpo class @, all of whose leaves 
are atomic classes, and in which the children of the non-atomic class Z are two 
proper subclasses J!, Y* C F such that J! U F? = GF. Because & is wpo its sub- 
classes satisfy the descending chain condition (Proposition 12.3.1), so this tree con- 
tains no infinite paths and thus is finite by Konig’s Lemma; its leaves are the desired 
atomic classes. i 


Thanks to the following result, the problem of computing growth rates of wpo 
classes can be reduced to that of computing growth rates of atomic classes. We leave 
the proof as an easy exercise for the reader. 


Proposition 12.3.21 The upper growth rate of a wpo permutation class is equal to 
the greatest upper growth rate of its atomic subclasses. 


The notion of representing permutation classes as ages raises several interesting 
questions. For instance, define {(A, B) as the set of all permutation classes that can be 
expressed as Sub(f : A > B). Given two linear orders A and B, we might ask if we can 
characterize T(A,B). Atkinson, Murphy, and Ruskuc [25] were the first to investigate 
this question; they characterized the classes of the form Sub(f : N > N), which they 
called natural classes. It is worth noting that every principal class is, up to symmetry, 
a natural class—sum closed classes are natural classes, while skew closed classes 
can be expressed as Sub(f : —N — N). Huczynska and RuSkuc [103] later studied 
classes of the form Sub(f : A — N) for arbitrary linear orders A, which they called 
supernatural classes. Of the many interesting questions that remain open, we quote 
their “contiguity question”: 


Question 12.3.22 (Huczynska and RuSkuc [103]) If @ € T(iN, N) NT(KN,N), is 
6 ET JN,N) for alli< j <k? 


A stronger version of the joint embedding property, called amalgamation, has 
also been considered. Let @ be a permutation class containing the permutation p. 
We say that @ is p-amalgamable if given two permutations 0,7 € @, each with a 
marked copy of p, we can find a permutation 7 € @ containing both o and T such 
that the two marked copies of p coincide. A class is called homogeneous if it is p- 
amalgamable for every p € @. Cameron [71] proved that there are precisely five in- 
finite homogeneous permutation classes: Av(12), Av(21), the layered permutations, 
the reversed layered permutations, and the class of all permutations. As Cameron 
noted, the permutation case turned out to be far simpler than several noteworthy 
cases considered before, such as the undirected graph case (established in 1980 by 
Lachlan and Woodrow [124]), the tournament case (due to Lachlan [123]), and the 
directed graph case (Cherlin [73]). 

We conclude this subsection by relating amalgamation and splittability. 


Proposition 12.3.23 (Jelinek and Valtr [105]) Every permutation class that is not 
1-amalgamable is splittable. 
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Proof. Suppose that the class @ is not 1-amalgamable, so there are permutations 
o,T € @ with marked entries o(i) and t(j) respectively such that no permutation in 
@ contains copies of o and t in which the entries corresponding to o(i) and t(j) 
coincide. We claim that @ is equal to the merge of its proper subclasses @ MN Av(o) 
and @ M Av(t). Let x € @ be arbitrary. Color an entry of 7 red if it can play the role 
of o(i) ina copy of o in 7, and blue otherwise. None of the red entries can play the 
role of t(j) and none of the blue entries can play the role of o(i), showing that the 
red entries avoid t and the blue entries avoid o, proving the result. a 


12.4 The set of all growth rates 


The final topic of this survey, the set of growth rates of all permutation classes, relies 
on the rich toolbox of structural results collected in the previous section as well as 
the theory of the three different flavors of grid classes introduced in Sections 12.4.1- 
12.4.3. Like the ideas of the previous section, growth rates have been studied for 
a wide variety of combinatorial objects, for example, posets, set partitions, graphs, 
ordered graphs, tournaments, and ordered hypergraphs. For details of the work on 
growth rates in this more general context, we refer to Bollobds’ survey for the 2007 
British Combinatorial Conference [43]. 

Characterizing the set of growth rates of permutation classes is easy at the very 
low end of the spectrum. The number of permutations of length n in a class must be 
an integer, and if it is ever 0 then the class is empty for all larger lengths. Thus there 
are no growth rates properly between 0 and 1. The next jump takes more work, and 
is referred to as the Fibonacci Dichotomy. 


Theorem 12.4.1 (Kaiser and Klazar [107]) Suppose that @ is a permutation class. 
If |Gn| is ever less than the nth Fibonacci number, then |G,| is eventually polynomial. 


The Fibonacci numbers referred to above are the combinatorial Fibonacci num- 
bers, which begin 1,1,2,3,5,... forn =0,1,2,3,4,.... Classes with this enumera- 
tion certainly exist. For example, both @{1,21} and O{1,12} are counted by the 
Fibonacci numbers. Huczynska and Vatter [104] gave a proof of Theorem 12.4.1 us- 
ing monotone grid classes, which we sketch in Section 12.4.1. We note in passing 
that while versions of Theorem 12.4.1 (i.e., jumps from polynomial to superpolyno- 
mial growth) are known to exist for a variety of different combinatorial structures 
(see Klazar [116] for numerous examples), the most general possible form of such a 
result remains open, as discussed in Pouzet and Thiéry [139]. 

It follows that the first three growth rates of permutation classes are 0, 1, and 9, 
where @ is the golden ratio. Next consider the classes @ Sub(k---21). It is easy to 
see that these classes are counted by (one version of) the k-generalized Fibonacci 
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Figure 12.21 
The set of all growth rates of permutation classes, as presently known. 


numbers, i.e., that they have generating functions of the form 


1 


l—x—x2 —--.— xk’ 


The growth rates of these classes are the largest roots of the polynomials x* — x*—! — 
---— 1, or equivalently (by multiplying by x — 1), the largest roots of the polynomials 
x'+l__9x* 4.1, These classes converge to the class of layered permutations, and thus 
their growth rates converge to 2. Kaiser and Klazar [107] showed that these are the 
only growth rates of permutation classes below 2, making 2 the least accumulation 
point of growth rates. Their work was later generalized by Balogh, Bollobas, and 
Morris [32], who showed that growth rates of ordered graph classes take on precisely 
the same values below 2 (every permutation class can be viewed as a class of ordered 
graphs, but not vice versa). 
This result has since been extended in Vatter [161] where the growth rates up to 


K = the unique real root of x? — 2x? — 12.21 


were characterized. One implication of this result is that growth rates of permutation 
classes and ordered graph classes diverge above 2. Sections 12.4.2 and 12.4.3 intro- 
duce the machinery (geometric and generalized grid classes) needed to study small 
permutation classes, while Section 12.4.4 sketches the proof of the following result. 


Theorem 12.4.2 (Vatter [161]) The sub-« growth rates of permutation classes con- 
sist precisely of 0, 1, 2, and roots of the families of polynomials (for all nonnegative 


k and ¢) 


e xl _ xk 4.1 (the sub-2 growth rates, identified by Kaiser and 
Klazar [107]), 


@ (2x? —1)xtth gx! +1, 


@ (x9 — 2x? — 1)x* +1 (accumulation points of growth rates which themselves 
accumulate at «), 


@ xx — x? —2x — 3, 0 — x4 — 3 — 2x? — 3x -1, 8 — x? — x - 3, and x4 - 


ex? -3x-1. 


Theorem 12.4.2 shows that « distinguishes itself on the number-line of growth 
rates by being the first accumulation point of accumulation points. It is also the 
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Figure 12.22 
An infinite antichain used to make an interval of growth rates. 


least growth rate at which one finds uncountably many permutation classes; indeed, 
Klazar [115] originally defined k as 


k = inf{y : uncountably many classes @ satisfy gr(@) < 7}. 


One direction follows without too much work—the growth rate of the downward clo- 
sure of the infinite antichain from Figure 12.17 is equal to k, so there are uncountably 
many classes of growth rate k. The other direction will be proved in Section 12.4.4. 
The phase transition at kK also has ramifications for exact enumeration: 


Theorem 12.4.3 (Albert, RuSkuc, and Vatter [16]) Every permutation class with 
growth rate less than kK is strongly rational. 


For this variety of reasons we call classes of growth rate less than k small. At 
the other end of the spectrum, Albert and Linton [14] constructed an uncountable 
set of growth rates of permutation classes, and conjectured that at some point the set 
of growth rates of permutation classes contains all subsequent real numbers. Let us 
define 


A =inf{y : every real number x > y is the growth rate of a permutation class}. 


By refining their techniques, Vatter [159] showed that A exists and is less than 2.49. 
Bevan [35] has since lowered this bound, showing that A < 2.36. Together, this result 
and Theorem 12.4.2 establish the number-line of growth rates shown in Figure 12.21. 

The results about A have been proved by constructing families of sum closed 
classes with a great deal of flexibility in their growth rates. For example, one of 
the constructions from [159] features the infinite antichain shown in Figure 12.22. 
Let A denote the members of this antichain of length at least 5. Clearly A contains 
two permutations of each of these lengths, both of which are sum indecomposable. 
We can compute the generating function for @ Sub(A) if we know how many sum 
indecomposable permutations of each length are properly contained in a member of 
A. This sequence can be shown to be 


(rn) = 1,1,3,5,6,6,6,6,... 


(starting at n = 1). Therefore the sequence counting sum indecomposable permuta- 
tions in @ Sub(A) is 
(tr) = 1,1,3,5,8,8,8,8,.... 
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Figure 12.23 


0 -i1 
A 1 -l 


by c) = 1, co =4, c3 = 10, and cq = 12, while the row divisions are r; = 1, r2 = 7, 
and r3 = 12. 


i -gridding of a permutation. Here the column divisions are given 


If we want to construct a subclass of GSub(A) we may choose any subset of A 
together with all of the sum indecomposable permutations properly contained the 
members of A. Thus for any sequence (s,) which satisfies r, < s, < ty for all n 
we can construct a permutation class with generating function 1/(1—)s,x"). The 
growth rates of such classes can be shown to consist of the entire interval of real 
numbers between approximately 2.49 and 2.51. 


12.4.1 Monotone grid classes 


As mentioned in Section 12.3.1, Stankova [150] computed the basis of the skew- 
merged permutations, 


Av(21) © Av(12) = Av(2143, 3412). 


This class is the prototypical example of a monotone grid class, although we need 
some notation to introduce these in general. 

The definition of monotone grid classes is quite similar to the definition of in- 
terval minors from Section 12.2.1, although here we are subdividing the plot of a 
permutation, rather than a matrix. Let 7 be a permutation of length n and choose 
intervals X,Y C [1,n]. We write 2(X x Y) to denote the permutation that is order 
isomorphic to those entries with indices from X and values from Y. 

Given at X u matrix M consisting of 0, 1, and —1 entries, an M-gridding of the 
permutation 7 of length n is a choice of column divisions 1 = cy <--: < ¢41) = 
n+1 and row divisions | =r, < --+ < 74; =n-+1 such that for all i and j, 
1([ci,ci41) X [rj,7j+1)) is increasing if M(i, j) = 1, decreasing if M(i, 7) = —1, and 
empty if M(i, 7) = 0. Figure 12.23 shows an example. The class of all permutations 
possessing M-griddings is called the monotone grid class of M, and denoted by 
Grid(M). 

Our first major result characterizes the classes contained in Grid(M) for some 
finite 0/+1 matrix M. We call such classes monotonically griddable (thus there are 
both monotone grid classes and monotonically griddable classes, and it is important 
to be recognizant of this distinction). First we need to establish an alternate charac- 
terization of monotone griddability. Recall from Section 12.3.2 that an axes-parallel 
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rectangle is any rectangle of the form X x Y for intervals X and Y. If R=X x Y is 
an axes-parallel rectangle, we let 7(R) denote the permutation which is order iso- 
morphic to the entries of 2 lying in R, that is, with indices in X and values in Y. 
The axes-parallel rectangle R is then nonmonotone for the permutation 7 if 7(R) is 
nonmonotone. We further say that the line L slices the rectangle R if L intersects the 
interior of R. Finally, a collection £ of lines and a collection ® of rectangles, we say 
that £ slices f if every rectangle in 9% is sliced by some line in £. 


Proposition 12.4.4 The class @ is monotonically griddable if and only if there is 
a constant £ such that for every permutation m © @, the collection of axes-parallel 
nonmonotone rectangles of 1 can be sliced by a collection of £ vertical or horizontal 
lines. 


Proof. — First, if @ C Grid(M) for a 0/41 matrix M of size t x u, then every TE @ 
has an M-gridding with at most t+ u—2 vertical and horizontal lines and by definition 
these lines must slice every nonmonotone rectangle of 7. 

For the other direction, suppose that there is a constant ¢ such that for every 7 € @ 
there is a collection £ of vertical and horizontal lines that slice every nonmonotone 
rectangle of 2. These lines define a monotone gridding of 7 of size t x u witht+u< 
£+2, i.e., they show that a € Grid(M) for some 0/+1 matrix of this size. There are 
only finitely many such matrices, so letting M® denote the direct sum of all such 
matrices we see that @ C Grid(M®),. a 


We can now state and prove the characterization of the monotonically griddable 
classes. One direction is clear. Using the notation 


6°21=21@---@21, 
—_--__—_——" 


a copies of 21 


it follows that if @¢21 € Grid(M) then M must have at least a rows and a columns. 
Analogously, 6712 cannot lie in the grid class of a matrix that is smaller than b x b. 
In other words, if @ is monotonically griddable, it must avoid 6°21 and 6?12 for 
some values of a and b. As we prove below, this is also a sufficient condition for 
monotone griddability. The proof we give relies on Stankova’s computation of the 
basis of the skew-merged permutations. 


Theorem 12.4.5 (Huczynska and Vatter [104]) A permutation class is monotoni- 
cally griddable if and only if it does not contain arbitrarily long sums of 2\ or skew 
sums of 12. 


Proof. | As we have already observed the other direction of the proof, let @ be a 
permutation class avoiding @421 and 6°12. We prove by induction on a+ b that 
there is a function @(a,b) such that given any permutation 7 € @, the nonmonotone 
rectangles of z can be sliced by a collection of (a,b) vertical or horizontal lines. 

If either a or b equals 1 then @ consists solely of monotone permutations and 
thus we may set £(1,b) = £(a,1) = 0. The next case to consider is a = b = 2, mean- 
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Figure 12.24 
The four regions in the proof of Theorem 12.4.5. 


ing that @ avoids both 2143 and 3412. Thus @ is a subclass of the skew-merged 
permutations, and so we may set (2,2) = 2. 

For the inductive step, we may assume by symmetry that a > 3 and b > 2. Take 
an arbitrary 2 € @. If x avoids 6721 = 2143 then its nonmonotone rectangles can be 
sliced by £(2,b) vertical and horizontal lines by induction. Thus we may assume that 
m contains 2143. We fix a specific copy of 2143 in z and then partition the entries of 
7 into the four regions shown in Figure 12.24. Clearly every nonmonotone rectangle 
either lies completely in one of these regions or is sliced by one of the four lines 
bordering these regions. Now notice that the entries in regions (I) and (III) avoid 
@*—!21, while those in regions (II) and (IV) avoid ©°-!12. This shows that we can 
set 

l(a,b) = 2€(a—1,b) +2€(a,b— 1) +4, 


completing the proof. | 


This result allows us to sketch the proof of the Fibonacci Dichotomy (Theo- 
rem 12.4.1). Suppose that @ is a permutation class and |@;,| is less than the nth 
Fibonacci number. The classes @{1,21} and O{1, 12} are both counted by the Fi- 
bonacci numbers. Therefore @ cannot contain either of these classes, and thus is 
monotonically griddable by Theorem 12.4.5. 

Next we recall the definition of alternations, first encountered in Section 12.3.2. 
An alternation is a permutation whose plot can be divided into two parts, by a single 
horizontal or vertical line, so that for every pair of entries from the same part there 
is an entry from the other part that separates them. A parallel alternation is one in 
which the two halves of the alternation form monotone subsequences, either both 
increasing or both decreasing, while for a wedge alternation one of these is increasing 
and the other is decreasing. It follows from the Erdés-Szekeres Theorem that every 
sufficiently long alternation contains a long parallel or wedge alternation. If @ were 
to contain arbitrarily long alternations of either type, its growth rate would be at least 
2. Thus there is a bound on the length of alternations in @. We now appeal to the 
following result. 


Proposition 12.4.6 (Huczynska and Vatter [104]) Suppose that @ is monotoni- 
cally griddable and that the length of alternations in @ is bounded. Then © C 
Grid(M) for a 0/41 matrix M in which no two nonzero entries share a row or col- 
umn. 
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This result shows that if @ has sub-Fibonacci enumeration, then it is contained 
in the monotone grid class of a “signed permutation matrix.” All that remains is to 
show that such classes have eventually polynomial enumeration. Both Kaiser and 
Klazar [107] and Huczynska and Vatter [104] did so by bijectively associating such 
classes with downsets of vectors in N’ and then appealing to a 1976 Monthly problem 
posed (and solved) by Stanley [151]. Homberger and Vatter [102] present a construc- 
tive proof that gives an algorithm for computing these polynomials. 

Monotone grid classes were first introduced (under a different name) in Murphy 
and Vatter [133], where the focus was on which monotone grid classes are wpo. Let 
M be a 0/+1 matrix of size t x u. The cell graph of M is the graph on the vertices 
{(i, j) : M(i, j) 4 0} in which (i, 7) and (k, 2) are adjacent if the corresponding cells 
of M share a row or a column and there are no nonzero entries between them in 
this row or column. We say that the matrix M is a forest if its cell graph is a forest. 
Viewing the absolute value of M as the adjacency matrix of a bipartite graph, we 
obtain a different graph, its row-column graph, which is the bipartite graph on the 
vertices X|,...,X7,V1,---,)u Where there is an edge between x; and y; if and only if 
M(i, j) 4 0. It is not difficult to show that the cell graph of a matrix is a forest if and 
only if its row-column graph is also a forest (a formal proof is given in Vatter and 
Waton [162]). 


Theorem 12.4.7 (Murphy and Vatter [133]) The grid class Grid(M) is wpo if and 
only if the cell graph of M is a forest. 


Theorem 12.4.7 has since been generalized in several directions. In the next sub- 
section we will see a generalization of the wpo part of this result (Theorem 12.4.15). 
Brignall [63] has presented another generalization of Theorem 12.4.7 in the context 
of the generalized grid classes of Section 12.4.3. 

Somewhat surprisingly, we know almost nothing about the bases of monotone 
grid classes. The skew-merged permutations have a finite basis, and Waton [163] 
showed that the monotone grid class of J2 (the 2 x 2 all-one matrix) has a finite basis. 
Thus we remain far away from the following. 


Conjecture 12.4.8 Every monotone grid class has a finite basis. 


We have used grid classes to establish the Fibonacci dichotomy, but what about 
growth rates of grid classes themselves? In exploring this question, Bevan [34] un- 
covered a beautiful connection between permutation patterns and algebraic graph 
theory. Recall that the spectral radius of a graph is the largest eigenvalue of its 
adjacency matrix. 


Theorem 12.4.9 (Bevan [34]) The growth rate of Grid(M) exists and is equal to the 
square of the spectral radius of the row-column graph of M. 


It is well beyond the scope of this survey to attempt to sketch Bevan’s proof, but 
we nonetheless attempt to convey some essence of his approach. Every connected 
component in the row-column graph of M corresponds to some submatrix of M, and 
we call this submatrix a connected component of M. It is not difficult to establish 
the following result, which was first used in Vatter [161]. 


Permutation Classes 807 


Figure 1228 ow 


Waton proved that the monotone grid class of the matrix ( ‘a ; ze ) is equal to 


the union of the two figure classes shown above. 


Proposition 12.4.10 The upper growth rate of Grid(M) is equal to the greatest up- 
per growth rate of the monotone grid class of a connected component of M. 


Spectral radii follow the analogous pattern—the spectral radius of the graph G 
is equal to the greatest spectral radius of a connected component of G. Therefore 
it suffices to consider grid classes with connected row-column graphs. A tour on a 
graph is a walk (a sequence of not-necessarily distinct vertices, each connected by 
an edge) that ends where it began. Bevan showed that if the row-column graph of 
M is connected, then the number of permutations of length n in Grid(M) is, up to a 
polynomial factor, equal to the number of balanced tours of length 2n in the row- 
column graph of G, where a balanced tour is a tour that traverses every edge the same 
number of times in each direction. He then showed that this balance condition does 
not affect the asymptotics of tours, establishing the theorem. 

As a consequence of Bevan’s Theorem 12.4.9, we may appeal to the significant 
literature on algebraic graph theory to determine the possible growth rates of mono- 
tone grid classes. First, obviously every growth rate of a monotone grid class is an 
algebraic integer, because it is the square of an eigenvalue of a zero/one matrix. More 
strikingly, if the row-column graph of M is a cycle, then the growth rate of Grid(M) 
is equal to 4, no matter how long the cycle. Also, if the growth rate of a monotone 
grid class is less than this, then it is equal to 4cos*(z/k) for some integer k > 3. 
Finally, for every growth rate y > 2+ \/5 ~ 4.24, there is a monotone grid class with 
growth rate arbitrarily close to y. 

The final question we address in this subsection is whether monotone grid classes 
are atomic. Many monotone grid classes (such as the class of skew-merged permuta- 
tions) are atomic. However, monotone grid classes are not, in general, atomic. In his 
thesis [163], Waton established necessary and sufficient conditions for a grid class to 
be atomic. Let M be a 0/+1 matrix. Given a cycle in its row-column graph, we say 
that the sign of the cycle is the product of the entries corresponding to its edges. So, 
for example, the matrix 

( 1 1 -1 ) 
a1: 1 


contains a positive cycle (corresponding to columns | and 3) and two negative cycles 
(corresponding to columns | and 2 and columns 2 and 3). Indeed, the grid class of 
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this matrix is not atomic; Waton showed that it is the union of the two figure classes 
shown in Figure 12.25. We conclude with his full characterization. 


Theorem 12.4.11 (Waton [163]) The monotone grid class Grid(M) is atomic if and 
only if the row-column graph of M does not have a connected component containing 
a negative cycle together with any other cycle. 


12.4.2 Geometric grid classes 


In Section 12.1.1 we introduced the class of permutations that can be drawn on an 
X and called it Sub(X), though to avoid cumbersome notation we denote it by 2 
here. We return to this class throughout this subsection because it is the prototypical 
example of a geometric grid class. Clearly the class 2 is a subclass of the skew- 
merged permutations, but it is not all of the skew-merged permutations. For example, 
the permutation 3142 cannot be drawn on an X because once we place the 3, 1, and 4 
on the X, there is no place for the 2 to lie simultaneously above the | and to the right 
of the 4: 


By symmetry, 2413 also cannot be drawn on an X, so 2 is also a subclass of the 
separable permutations. Indeed, it is not hard to see that 2 is the class of skew- 
merged separable permutations, 


X = Av(2143,2413,3142, 3412), 


because every permutation drawn on an X must have some point that is at least as far 
away from the center of the X as every other point. Therefore every permutation in 
& is of the form 162, 761,107, or 701 for some a € 2. This leads quickly 
to the generating function of the class. From the above, we have 


K =MuI@eX)u(L @vyuloX)u(L ol). 


Moreover, (16 2)N(2% @1)=(18 & G1), so we see (by symmetry) that if f is 
generating function for nonempty permutations in 2°, f =x-+4xf — 2x’ f. Thus the 


generating function for 2% is 
x 


1—4x+2x2° 


It follows that the growth rate of 2 is 2+ /2 = 3.41. This is significantly lower 
than the growth rate of the skew-merged permutations (4, which follows from Atkin- 
son [20] or Bevan’s Theorem 12.4.9) or that of the separable permutations (approxi- 
mately 5.83, as we saw in Section 12.1.1). 


Permutation Classes 809 


Figure 12.26 
The permutation 6327415 lies in the geometric grid class of the matrix 


-1 1 1 
0 -1 -1 /° 


aaa 


Figure 12.27 
“Straightening” a member of a monotone grid class to show that it also lies in the 
corresponding geometric grid class. 


Suppose that we have a 0/+1 matrix M, as in the definition of a monotone grid 
class. The standard figure of M, denoted Ay, is the figure in R* consisting of two 
types of line segments for every pair of indices i, j such that M(i, j) 4 0: 


e the increasing open line segment from (i— 1, j— 1) to (i, j) if M(i, j) = 1 or 
e the decreasing open line segment from (i-— 1, j) to (i, j— 1) if M(i, 7) =—1. 


The geometric grid class of M, denoted Geom(M), is the set of all permutations 
that can be drawn on Ay (an example is shown in Figure 12.26). The inclusion 
Geom(M) C Grid(M) always holds, but our example of .2° shows that the geometric 
grid class of M may be a proper subclass of its monotone grid class. Indeed, this 
happens precisely when M contains a cycle: 


Proposition 12.4.12 The classes Grid(M) and Geom(M) are equal if and only if the 
cell graph of M is a forest. 


Figure 12.27 gives a sense of how to prove Proposition 12.4.12. Suppose that the 
cell graph of M is a tree (the forest case follows easily from the tree case) and choose 
one cell to be the root. Now take the plot of any gridded permutation z € Grid(M) 
and stretch (or shrink) this row vertically so that the points lie on a line of slope +1. 
Next, for every neighbor cell in the same row (there can be at most two), we stretch 


810 Handbook of Enumerative Combinatorics 


the x-axis, while for every neighbor cell in the same column we stretch the y-axis. 
Because M is a tree, we can continue this process throughout all cells without having 
to revisit a cell. 

Next we turn to the enumeration of geometric grid classes, where it turns out we 
can say a lot more than we could for monotone grid classes. First, though, we need 
a bit more precision. There are infinitely many ways to draw every permutation 7 € 
Geom(M) on the standard figure Ajj because we can move the points by tiny amounts 
without changing the underlying permutation, but clearly we want to consider minute 
changes such as this as equivalent. Therefore we say that a gridded permutation 
is a (drawing of a) permutation together with grid lines corresponding to M, and 
we denote the set of all gridded permutations in Geom(M) by Geom*(M). Every 
permutation in Geom(M) then corresponds to at least one gridded permutation in 
Geom*(M), and not more than iF Me ‘J ey) , because there are only so many places we 
can insert the grid lines. Thus we obtain the following. 


Observation 12.4.13 The (upper, lower, and proper) growth rates of Geom(M) and 
Geom*(M) are identical. 


We would like to describe an encoding of the gridded permutations in Geom’ (M) 
by words over a finite alphabet. First let us return to the permutations drawn on an 
X to see an easy example before presenting the general construction. Below is our 
drawing from Figure 12.4 with two changes: First, the points are labeled by what 
quadrant they lie in (we consider the center of the X to be the origin), and second, 
the line segments have been assigned orientations. 


To encode this gridded permutation, we order these points according to their distance 
from the beginning of their line segment, which in this case is also their distance from 
the center of the figure, and record the labels of the points in this order. While it may 
take a ruler to verify it, in our example above the encoding is 21242443. 

Suppose that we have an arbitrary 0/+1 matrix M. The important property of 
the orientation above is that it is consistent—in each column all lines are oriented 
either left or right, and in each row all lines are oriented either up or down. This is 
not possible for all matrices, for example, the standard figure of the matrix 


1 1 

1 -l 
cannot be consistently oriented. But there is a way to remedy this. The grid lines of 
the standard figure consist of x =i and y =/ for all (relevant) integers i. If we also 


add grid lines at half-integer values of x and y, each cell is chopped into four but the 
geometric grid class itself is not changed. 
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In terms of the matrix we started with, this operation is equivalent to performing 
the substitutions 


0+ (5 oo) ie at Te : en 

and we call the resulting matrix, written M x2 the double refinement of M. Thus the 
standard figure of M*? consists of 2 x 2 blocks that are all subfigures of the X, and 
thus we can use its consistent orientation to define a consistent orientation of M*?. 

For the rest of this discussion let us suppose that we have a consistent orienta- 
tion for the standard figure of M (replacing M by M*? if necessary). Next we fix 
a cell alphabet, ©, containing one letter corresponding to each nonempty cell of 
M. Finally, we encode a gridded drawing (of a permutation) by ordering its points 
by their distance from the beginning of their line segment (we can assume there are 
no ties by possibly moving points by a minuscule amount) and then recording the 
corresponding “cell letter” of each point in this order. 

Formally, we have just defined a map 


g* : Xiy —> Geom*(M). 


This is in general a many-to-one map, because the letters of Ly often “commute”; 
for example, in our encoding of permutations drawn on an X, interchanging adjacent 
occurrences of 1 and 3 does not change the image (the gridded permutation), and 
the same holds for 2 and 4. This is because the corresponding cells of the standard 
figure are independent, meaning that they share neither a row nor a column. For 
general matrices M, we write v =y w if w can be obtained from v via a sequence of 
interchanges of adjacent occurrences of letters corresponding to independent cells. 
It is not difficult to see that g* is actually a bijection when restricted to equivalence 
classes of words modulo =y, i.e., that the map 


gy’ : Xt,/ =u— Geom*(M) 


is a bijection. 

Objects such as £* / = are known as trace (or partially commutative) monoids. 
They were first introduced by Cartier and Foata [72] in 1969 who used them to give a 
combinatorial proof of MacMahon’s Master Theorem. Cartier and Foata showed via 
MObius inversion that the generating function for equivalence classes of £*/ = (by 
length) is 

1 
+ 03x3 — c4xt + +++)? 


2 


1 — (cyx — cox 


where c; is the number of k-elements subsets of X that pairwise commute (see also 
Flajolet and Sedgewick [86, V.3.3]). In our example of permutations drawn on an X, 
we have that cp = 1, cy = 4, co = 2, and cz = cq = 0. Thus the generating function 
for the corresponding trace monoid is 


1 
1—4x+2x2° 
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Figure 12.28 
Two very different griddings of the permutation 564312. 


As Observation 12.4.13 tells us should be the case, this generating function has the 
same growth rate as the class 2”. 

Growth rates of trace monoids were studied by Goldwurm and Santini [96]. Be- 
van [37] used their work to express the growth rate of Geom(M) in terms of the 
row-column graph of M, giving a geometric analogue of his Theorem 12.4.9. A k- 
matching of a graph is a set of k edges, no two incident with the same vertex. Letting 
m,(G) denote the number of k-matchings of a graph on n vertices, its matching poly- 


nomial is defined by 
uig(x) = Y (—1)km(G)x*. 
k>0 


(No matching can contain over |n/2| edges, so this is indeed a polynomial.) We state 
Bevan’s result below in terms of the double refinement of M because this is necessary 
in order to guarantee that we have a consistent orientation. 


Theorem 12.4.14 (Bevan [37]) The growth rate of Geom(M) exists and is equal to 
the square of the largest root of the matching polynomial of the row-column graph of 
M*?. 


In contrast to Theorem 12.4.9, this result shows that changing the sign of an 
entry of M can change the growth rate of the corresponding geometric grid class. For 
example, Bevan [37] notes that 


g(Geom( “| } “| ))=4<3+v2=er(Geom( | } “| )). 


Having addressed the asymptotic enumeration of geometric grid classes, we 
move on to their exact enumeration. As in our study of monotonically griddable 
classes, we are interested not only in geometric grid classes themselves, but also 
in their subclasses. Thus we say that the class @ is geometrically griddable if 
@ C Geom(M) for some finite 0/1 matrix M. Unlike the case with monotoni- 
cally griddable classes, there is no known characterization of geometrically griddable 
classes. 

With exact enumeration we have a new type of problem. While our map @? re- 
stricts to a bijection between £4,/ =y and Geom*(M), a given permutation may 
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Figure 12.29 

The simple permutations in Av(3124,4312) lie in the union of the two geometric 
grid classes shown on the left; the intersection of these two classes is the geometric 
grid class shown on the right. 


have many different griddings, as demonstrated in Figure 12.28. In [4], this issue is 
addressed by imposing an order on all Geom* (M) griddings of a given permutation 
mz. Among all of these griddings, we would like to select the minimal one, called the 
preferred gridding. By using a trick involving “marking entries” it can be shown 
that the set of all preferred griddings of permutations in a given geometrically grid- 
dable class is in bijection with a regular language, establishing the following result. 


Theorem 12.4.15 (Albert, Atkinson, Bouvel, RusSkuc, and Vatter [4]) Every geo- 
metrically griddable class is strongly rational and finitely based. 


In particular, every geometrically griddable class is wpo (we have rewritten 
history a bit here; the wpo property plays an important role in the proof of The- 
orem 12.4.15, and thus must be established first). By Proposition 12.4.12, Theo- 
rem 12.4.15 thereby generalizes one direction of Theorem 12.4.7. 

Continuing in this line of research, Albert, RuSkuc, and Vatter studied the infla- 
tions and substitution closures of geometrically griddable classes. Their main result 
is the following generalization of Albert and Atkinson’s Theorem 12.3.11. 


Theorem 12.4.16 (Albert, Ruskuc, and Vatter [16]) Let @ be a geometrically 
griddable class. Its substitution closure, (@), is strongly algebraic. Moreover, for 
every strongly rational class WY, the inflation @|Y| is strongly rational. 


It follows that all such classes are wpo. The proof of this result is well beyond the 
scope of this survey. Both parts of the theorem rely on the notion of “query-complete 
sets of properties” introduced by Brignall, Huczynska, and Vatter [65] in their gen- 
eralization of Albert and Atkinson’s Theorem 12.3.11. In addition, the proof of the 
second part of the theorem relies on the work of Brignall [61] on decompositions 
of permutations contained in inflations of the form @[%] and the work of Albert, 
Atkinson, and Vatter [8] on strongly rational classes (in particular, that the sum in- 
decomposable permutations in a strongly rational class themselves have a rational 
generating function). 

Theorem 12.4.16 greatly expands the applicability of the substitution decom- 
position, and has found numerous applications. One example is the work of Pan- 
tone [135], who used this approach to enumerate the class Av(3124,4312). Pantone 
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Figure 12.30 
Geometric descriptions of the 321-avoiding permutations and the simple skew- 
merged permutations (up to symmetry). 


showed that the simple permutations in this class lie in the union of the first and 
second geometric grid classes shown in Figure 12.29. He then counted the class of 
interest by computing the intersection of these two geometric grid classes (shown on 
the right of this figure; here the dot is a single point), constructing regular languages 
in bijection with the simple permutations in all three geometric grid classes, and then 
applying the machinery of the substitution decomposition. Three simpler examples 
in the same spirit are described in Albert, Atkinson, and Vatter [9]. 

Lately, though no general theory has been established, there has been some 
promising work done using geometric grid classes of infinite matrices. Recall Wa- 
ton’s Proposition 12.1.5, which shows that Av(321) is the class of permutations that 
can be drawn on two parallel lines. By taking these lines to have slope 1 and adding 
the grid lines x = i and y =i for all integers i, we see that Av(321) is the infinite 
geometric grid class shown on the left of Figure 12.30. This viewpoint, known as 
the staircase decomposition has been used by Guillemot and Vialette [97] in their 
study of the complexity of the permutation containment problem, by Albert, Atkin- 
son, Brignall, RuSkuc, Smith, and West [5] in their study of growth rates of classes 
of the form Av(321, 8), and by Albert and Vatter [17] in their study of 321-avoiding 
simple permutations. This latter work has since been refined by Bona, Homberger, 
Pantone, and Vatter [55] to enumerate the involutions in the classes Av(1342) and 
Av(2341). 

Albert and Vatter [17] also gave a geometric interpretation of the skew-merged 
permutations. It can be shown that every simple skew-merged permutation is, up to 
symmetry, an element of the infinite geometric grid class shown on the right of Fig- 
ure 12.30, and they used this fact to give a more structural derivation of the generating 
function for skew-merged permutations, originally due to Atkinson [20]. Albert and 
Brignall [11] studied the class of permutations whose corresponding Schubert va- 
rieties are defined by inclusions. They showed that the simple permutations of this 
class lie in an infinite sequence of geometric grid classes that they called crenella- 
tions (see Figure 12.31), and used this insight to enumerate the class. 
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Figure 12.31 
A crenellation. 


12.4.3 Generalized grid classes 


In a generalized grid class the cells are allowed to contain nonmonotone permu- 
tations. Formally, let W be at x u matrix of permutation classes. An .@-gridding 
of the permutation 7 of length n is a choice of column divisions 1 = cy <--- < 
Crt] =n-+1 and row divisions 1 =r) < +++ < 744; =n+1 such that for all i and j, 
W([ci,ci+1) X [rj,7j+1)) lies in the class “%,;. 

The class of all permutations with .#@-griddings is the grid class of .@, denoted 
Grid(.@). In the context of monotone grid classes, we defined the class @ to be 
monotonically griddable if @ C Grid(M) for some finite 0/+1 matrix M. Now we 
define the class @ to be Y-griddable if @ C Grid(.#) for some finite matrix ./@ 
whose entries are all subclasses of Y (for the purposes of this definition, we may take 
them all to be equal to ¥). From this perspective, a class is monotonically griddable if 
and only if it is (Av(21) U Av(12))-griddable (though this fact takes a bit of thought). 

As Theorem 12.4.5 characterizes monotonically griddable classes, our next result 
characterizes generalized grid classes. 


Theorem 12.4.17 (Vatter [161]) The permutation class © is G-griddable if and 
only if it does not contain arbitrarily long sums or skew sums of basis elements of 
G, that is, if there exists a constant m so that @ does not contain B, @---® By or 
Bi ©--- © By for any basis elements By,..., Bm of G. 


We prove this result by appealing to a result of Gyarfas and Lehel from 1970, 
but a proof from first principles can be found in Vatter [161]. One direction is again 
clear—if @ contains arbitrarily long sums or skew sums of basis elements of ¥, then 
it is not Y-griddable. 

Now suppose that @ does not contain arbitrarily long sums or skew sums of basis 
elements of Y and consider the set 


KR = {axes-parallel rectangles R : 7(R) ¢ G}. 


Thus for every R € Wz, 2(R) contains a basis element of Y, so in any Y-gridding 
of 7 every rectangle in 8, must be sliced by a grid line. Clearly the converse is 
also true (if every rectangle in iz is sliced by a grid line, these grid lines define a 


816 Handbook of Enumerative Combinatorics 


¢-gridding of 1), so @ is Y-griddable if and only if there is a constant such that, 
for every 2 € @, the set Wz can be sliced by @ horizontal or vertical lines. (This is 
the analogue of Proposition 12.4.4 for generalized grid classes.) 

We say that two rectangles are independent if both their x- and y-axis projec- 
tions are disjoint, and a set of rectangles is said to be independent if they are pairwise 
independent. An increasing sequence of rectangles is a sequence Rj,...,R,, of in- 
dependent rectangles such that R;;; lies above and to the right of R; for all i > 0. 
Decreasing sequences of rectangles are defined analogously. By our assumptions on 
@, there is some constant m such that for every 7 € @ the set 9%, contains neither an 
increasing nor a decreasing sequence of size m. 

An independent set of rectangles corresponds to a permutation. Therefore, by 
the Erdés-Szekeres Theorem, 9%, cannot contain an independent set of size greater 
than (m—1)* +1. The proof of Theorem 12.4.17 is therefore completed by the fol- 
lowing result of Gyarfaés and Lehel, which was later strengthened by Karolyi and 
Tardos [109]. 


Theorem 12.4.18 (Gyarfas and Lehel [99]) There is a function f(m) such that for 
any collection Kt of axes-parallel rectangles in the plane that has no independent set 
of size greater than m, there exists a set of f(m) horizontal and vertical lines that 
slice every rectangle in KR. 


Our next question is which permutation classes can be gridded by one of their 
proper subclasses. We call the class @ grid irreducible if it is not Y-griddable for 
any proper subclass ¥Y C @. Somewhat surprisingly, this question has a concrete 
answer. 


Proposition 12.4.19 The class © is grid irreducible if and only if @ = {1} or if for 
every 1 € © either @Sub(z) C © or GSub(z) C @. 


Proof. | One direction is immediate from Theorem 12.4.17: if @ contains arbitrarily 
long sums or skew sums of all of its elements, then it is grid irreducible. Moreover, 
the only finite grid irreducible class is {1}, which takes care of that part of the other 
direction. 

Now suppose that @ is an infinite permutation class. If there is some 7 € @ 
such that neither & Sub(z) nor G Sub(z) is contained in @ then @ is (@ MN Av(z))- 
griddable by Theorem 12.4.17, and thus is not grid irreducible. | 


As shown in the proof of this result, if neither & Sub(z,) nor © Sub(7) is con- 
tained in @ then it is (@ M Av(7) )-griddable. We may then apply this to @ MN Av(z)) 
to see that if neither @Sub(z2) nor GQ Sub(z2) is contained in @ NM Av(71), it is 
(@ M Av(%1, %2))-griddable. Continuing in this manner, we can construct a descend- 
ing chain of classes 


C 2D CNAv(m™m) D OCNAvV(,m) 
I I l| 
ZO > (1) > (2) 


tU 


$U 
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such that @ is @“) -griddable for all i. Because there are infinite strictly descending 
chains of permutation classes, there is no guarantee that this process will terminate 
(see Vatter [161] for such a construction). However, if @ is wpo then it satisfies the 
descending chain condition by Proposition 12.3.1 and thus this process must stop. 
This proves the following. 


Proposition 12.4.20 If the class @ is wpo then it is Y-griddable for the grid irre- 
ducible class 


G ={1 : either @ Sub(a) C @ or GSub(z) C G}. 


We need a final result about generalized grid classes, which shows that atomic 
grid irreducible classes are very constrained. 


Proposition 12.4.21 Suppose that the class @ is atomic. Then it is grid irreducible 
if and only if it is sum or skew closed. 


Proof. Theorem 12.4.17 shows that every sum or skew closed class is grid ir- 
reducible, so it suffices to prove the reverse direction. Suppose that @ is atomic, 
but neither sum nor skew closed. Thus there are permutations 7,0 € @ such that 
@ Sub(z),© Sub(t) Z @. Because @ is atomic, we can find a permutation Tc @ 
containing both z and o. Thus @ does not contain arbitrarily long sums or skew 
sums of T, so it is (@ M Av(t))-griddable (again by Theorem 12.4.17). This shows 
that @ is not grid irreducible, completing the proof. | 


We conclude by investigating a special case of generalized grid classes and 
their relation to the splittability question. We call the 2 x 1 generalized grid class 
Grid(@ E) the horizontal juxtaposition of the classes Y and E (the obvious symme- 
try of this construction is called a vertical juxtaposition). Atkinson [21] introduced 
juxtapositions and established the structure of their basis elements. 

Using our results on generalized grid classes we are able to completely charac- 
terize the classes which can be expressed as nontrivial juxtapositions. If @ is not 
atomic then @ C WUE for proper subclasses J, E C @, so @ is also contained in 
the nontrivial juxtaposition Grid(Y E). Suppose instead that @ is atomic. If @ is 
either sum or skew closed then it is grid irreducible by Proposition 12.4.19 and thus 
not contained in a juxtaposition of proper subclasses. Otherwise, Proposition 12.4.21 
shows that @ is Z-griddable for a proper subclass J C @. Choose -#@ to be a matrix 
of minimal possible size such that all of its entries are equal to Y and @ C Grid(.-Z@). 
We may suppose by symmetry that .@ has at least two columns, and thus by slicing 
it by a vertical line we see that @ is contained in the horizontal juxtaposition of two 
proper subclasses, establishing the following. 


Proposition 12.4.22 A permutation class is contained in the juxtaposition of two 
proper subclasses if and only if it is neither sum nor skew closed. 


In particular, if a class is contained in the juxtaposition of two proper subclasses 
then it is splittable. Thus the splittability question is only interesting for sum or skew 
closed classes that are not substitution closed. 
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12.4.4 Small permutation classes 


Our goal in this subsection is to outline the proofs of Theorem 12.4.2, which specifies 
all possible growth rates of small permutations classes, and 12.4.3, which states that 
all small permutation classes are strongly rational, while glossing over some of the 
more technical details. Adopting a perspective that is slightly historically backward, 
we prove Theorem 12.4.3 first, by showing that every small permutation class is the 
inflation of a geometric grid class by a strongly rational class, and is thereby strongly 
rational itself. We then show that if @ is a small permutation class, its growth rate 
is either equal to 0, 1, or 2, or is equal to the growth rate of the largest sum or 
skew closed class contained in it (thereby establishing that these classes have proper 
growth rates). The actual list of possible growth rates provided in Theorem 12.4.2 can 
then be produced by characterizing all possible enumerations of sum indecomposable 
permutations in classes with growth rates less than k, though we do not go into that 
rather lengthy undertaking here. 

For the moment, let y be an arbitrary real number and suppose we would like to 
grid all permutation classes of growth rate less than y. That is, we would like to find 
a single cell class Y such that every permutation of (upper) growth rate less than ¥ is 
¢-griddable. 

To this end, define 


G = {mn : either gr (GSub(z)) < yor gr(QSub(z)) < y}. 


If x € Y, then at least one of @ Sub(z) or G Sub(z) has growth rate less than ¥, so 
in order to grid all permutation classes of growth rate less than y we must have 7 in 
our cell class. Our next result shows that this cell class can indeed be used to grid all 
classes of growth rate less than y. 


Proposition 12.4.23 For every real number ¥, if the permutation class © satisfies 
gt(@) < ¥ then it is G-griddable. 


Proof. Suppose to the contrary that @ satisfies gr(@) < y but is not %-griddable, 
and let f denote the generating function of @. Now fix an arbitrary integer m. We 
seek to derive a contradiction by showing that f(1/7) > m, which, because m is 
arbitrary, will imply that gt(@) > y. By Theorem 12.4.17 and symmetry, we may 
assume that there is a permutation of the form 8; @---@ B,, contained in @ where 
each f; is a basis element of 4. If gr(@B;) were less than y then B; would lie in %, 
so we know that gr(€)B;) > y for every index i. 

Let s; denote the generating function for the nonempty sum indecomposable per- 
mutations contained in (or equal to) B;, so that the generating function for @P; is 
given by 1/(1—s;). These generating functions have positive singularities that are 
less than or equal to 1/y, and because each s; is a polynomial this singularity must 
be a pole. Moreover, each s; has positive coefficients, which implies that the unique 
positive solution to s;(x) = 1 occurs for x < 1/y. In particular, s;(1/y) > 1 for all 
indices i. 

Now consider the set of permutations of the form a @---@ a for some k < m, 
where for each i, ; is a nonempty sum indecomposable permutation contained in 
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B;. Clearly this is a subset (but likely not a subclass) of @. Moreover, the generating 
function for this set of permutations is 


S} +8182 +++ +S 1°°°Sm, 


which is at least m when evaluated at 1/y. This implies that the generating function 
for @ is also at least m when evaluated at 1 /, completing the contradiction. | 


Proposition 12.4.23, which was not used in the original proof of Theorem 12.4.2, 
gives us an easily computable membership test to determine whether 7 € %. The 
small permutation classes are all G,_¢-griddable for some € > 0 so we begin by 
establishing some restrictions on Ye. 

The reader may have noticed that both infinite antichains we have encountered in 
this survey (in Figures 12.11 and 12.17) consist of variations on a single theme. While 
that particular coincidence should not be taken too far (there are many more infinite 
antichains based on different motifs, for example, all those lying in monotone grid 
classes with cycles), the theme of those two antichains plays a significant role in the 
characterization of small permutation classes. We define the increasing oscillating 
sequence as 

4,1,6,3,8,5,...,2k+2,2k—1,.... 


(This sequence, itself listed as A065164 in the OEIS [157], also arises in the study 
of juggling and genomics, see Buhler, Eisenbud, Graham, and Wright [68] and 
Pevzner [136], respectively.) 

An increasing oscillation is defined to be any sum indecomposable permutation 
that is order isomorphic to a subsequence of the increasing oscillating sequence. It is 
easily seen that there are only two increasing oscillations of each length, which are 
inverses of each other. Finally, a decreasing oscillation is the reverse of an increasing 
oscillation, and an oscillation is either an increasing or a decreasing oscillation. 

Let @ denote the downward closure of the set of all oscillations, 


@ = Sub(increasing and decreasing oscillations of all lengths). 


It can be shown that their substitution closure (@) has a finite basis (Vatter [161, 
Proposition A.2]). From this, it can then be computed that for every basis element B 
of (@), both @ Sub(f) and © Sub(B) have growth rates greater than 2.24. Therefore 
Proposition 12.4.23 immediately gives us the following. 


Proposition 12.4.24 The cell class GY, is contained in (@). 


This gives us a place to start, but we need to get some control on the structure of 
small permutation classes. In particular, we need an analogue of Proposition 12.4.6 
that will allow us to “chop” the griddings of these classes. The direct analogue of 
Proposition 12.4.6 is below, while Figure 12.32 shows why we must impose these 
hypotheses. 


Theorem 12.4.25 (Vatter [161, Theorem 5.4]) Suppose that the cell class Y con- 
tains only finitely many simple permutations and has bounded substitution depth. 
If the class @ is Y-griddable and the length of alternations in @ is bounded, then 
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Figure 12.32 

Two permutations that show why the hypotheses of Theorem 12.4.25 are necessary 
On the left is a long increasing oscillation, while on the right is a permutation with 
large substitution depth, each with an extra grid line. Both situations require a large 
number of additional grid lines in order to grid them into independent rectangles. 


€ CGrid(.H) for a matrix M in which every nonempty entry is a subclass of Y and 
no two nonempty entries share a row or a column. 


Clearly Theorem 12.4.25 is not enough for our purposes, since small permutation 
classes may contain arbitrarily long alternations. Still, the examples of Figure 12.32 
demonstrate obstructions to chopping any sort of gridding, and the actual chopping 
result used for small permutation classes (Theorem 5.4 of [161]) has precisely the 
same conditions on Y. Thus we need to verify that our cell classes Y-_¢ have finitely 
many simple permutations and bounded substitution depth. 

We tackle substitution depth first. It can be shown (Proposition 4.2 of [161]) 
that every permutation of substitution depth at least 81 contains a wedge alternation 
of length at least n. Furthermore, using the membership test provided by Proposi- 
tion 12.4.23 one can prove that for every y < 1+ @ © 2.62, the cell class Y, does not 
contain arbitrarily long wedge alternations. Thus there is some constant d such that 
every permutation in Y has substitution depth at most d. 

It is not difficult to establish that the growth rate of @ is precisely «. Moreover, 
using the fact that @ is the union of two posets (corresponding to the increasing and 
decreasing oscillations) neither of which has an antichain containing more than three 
elements, it follows that for every € > 0 we have Yee C(O’ ey for some k, where 


O* = Sub(oscillations of length at most k). 


With these two computations handled, we can then chop the griddings of small 
permutation classes to obtain the following result. 


Theorem 12.4.26 (Vatter [161, Theorem 5.4]) Suppose that gt(@) < «—€ for 
some € > 0. Then @ C Grid(.@) for a finite matrix M such that 


(1) every entry is equal to G_e_¢, the class of monotone permutations, or is empty, 


(2) every nonempty entry that shares a row or column with another nonempty entry 
is equal to the class of monotone permutations, and 


(3) no nonempty entry shares a row or column with more than one other nonempty 
entry. 
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Moreover, we have seen above that %_¢ is a subclass of (@*) and contains only 
permutations of substitution depth at most d. This gives us the following. 


Proposition 12.4.27 For every € > 0, the cell class G__¢ is strongly rational. 


Proof. Recall that by the second part of Theorem 12.4.15, the inflation of a geomet- 
rically griddable class by a strongly rational class is itself strongly rational. Choose 
d so that every permutation in Y._¢ has substitution depth at most d. Thus each of 
these permutations is either the inflation of a simple permutation by permutations of 
substitution depth at most d — 1 or the sum or skew sum of such permutations. Thus 
if we define 

G* = 6* UAv(21) UAV(12), 


we see that eae 
Geel OO |--J] 
- i 
d copies of G* 


Because (* is the union of a finite class with the class of all monotone permuta- 
tions, it is geometrically griddable, and thus the strong rationality of Y._¢ follows 
by iteratively applying the second part of Theorem 12.4.15. a 


It is possible to show the stronger result that % is strongly rational, but this 
requires more work, and Proposition 12.4.27 is enough for our purposes. The first 
of these purposes is to prove Theorem 12.4.3, which states that small permutation 
classes are strongly rational. By Theorem 12.4.26, every small permutation class is 
contained in Grid(M)|%_¢] for a matrix M in which no nonzero entry shares a row 
or column with more than one other nonzero entry. It follows that M is a forest so 
Grid(M) = Geom(M) by Proposition 12.4.12. The result now follows by the second 
part of Theorem 12.4.16. 

Our last goal is the characterization of growth rates of small permutation classes 
(Theorem 12.4.2). We observed in Proposition 12.4.10 that the upper growth rate of 
Grid(M) is equal to the greatest upper growth rate of the monotone grid class of a 
connected component of M. A similar argument holds for generalized grid classes 
and their subclasses. Thus given a class @ C Grid(.@), its upper growth rate is equal 
to the upper growth rate of the restriction of @ to a connected component of .@. We 
can now sketch the reduction to sum closed classes. 


Theorem 12.4.28 If the permutation class @ satisfies gt(@) < K then gr(@) exists 
and is equal to 0, 1, 2, or the growth rate of a subclass that is either sum or skew 
closed. 


Proof. Suppose that st(@) < K and set @° = @. By our previous work, we can 
find a cell class Y? C @NGx_¢ for some € > 0 such that @ is Y°-griddable. Next 
we can find a matrix .W° with all entries equal to subclasses of Y° and satisfying the 
conclusions of Theorem 12.4.26 such that @° C Grid(.W@°). By our remarks above, 
the upper growth rate of @° is the upper growth rate of the restriction of @° to a 
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connected component of .W@°. This connected component is either a single cell (in 
which case we get a subclass of Y°) or a pair of monotone cells. It can be shown that 
all subclasses of a 1 x 2 monotone grid class have growth rates 0, 1, or 2, so we are 
done if the latter situation holds. Thus we may assume the former situation holds and 
choose @! C Y° such that gr(@°) = gr(@'). 

Importantly, Proposition 12.4.27 shows that @! is strongly rational, and thus 
wpo. By Proposition 12.3.21, this implies that the upper growth rate of @! is equal 
to that of one of its atomic subclasses, say 7! C @!. By Proposition 12.4.20, we 
may now choose a grid irreducible class Y! such that o/! is Y'-griddable. Then we 
choose a matrix .@! with all entries equal to subclasses of Y!, which satisfies the 
conclusions of Theorem 12.4.26 and with o/! C Grid(.#'). The upper growth rate 
of ./! is then equal to that of a restriction of ./! to a connected component of ./!. 
We are done as before if this component consists of two cells, so we may assume that 
or(/!) = of(@") for a subclass G7 C Y!. 

By repeating this process indefinitely, we either find that the upper growth rate of 
€ is equal to 0, 1, or 2, or we construct an infinite descending chain 


C=PDP DE DA DF oP VDL WF DS*2dD.-.., 


all with identical upper growth rates. Moreover, because Y° is wpo, Proposi- 
tion 12.3.1 shows that this chain must terminate. Thus there is some 7 such that 
€' = of' =’, This implies that the class @! is both atomic and grid irreducible, 
and thus it is either sum or skew closed by Proposition 12.4.21, proving the theorem. 


Thus we have reduced the characterization of growth rates of small permuta- 
tion classes to the characterization of growth rates of small sum closed permutation 
classes, as promised. From this point Theorem 12.4.2 follows after a fairly technical 
analysis of sum indecomposable permutations. 
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13.1 Introduction 


The notion of parking functions was introduced by Konheim and Weiss [53] as a 
colorful way to describe their work on computer storage. The parking problem can 
be stated as follows. There are n cars C),...,C, that want to park on a one-way street 
with ordered parking spaces 0,1,...,n— 1. Each car C; has a preferred space a;. The 
cars enter the street one at a time in the order C},C2,...,C,. A car tries to park in its 
preferred space. If that space is occupied, then it parks in the next available space. If 
there is no space then the car leaves the street. The sequence (a1,...,d,) is called a 
parking function of length n if all the cars can park, i.e., no car leaves the street. 

It is easy to see that a sequence (a ,...,@,) is a parking function if and only if 
it has at least 7 terms less than i, for each 1 <i <n. Let aq) S a2) S++ S aq) 
be the non-decreasing rearrangement of terms a;’s, where ay;) is called the ith order 
statistics of (a1,...,@,). Then the sequence (a1,...,@,) is a parking function if and 
only if 0 < aj <i. * Another equivalent definition is that a sequence (a1,...,4n) isa 
parking function if and only if there is a permutation o € G, such that 0 < agi) <i. 

The number of parking functions of length n is (n+ 1)"~', a result obtained in 
the very first paper [53] on the subject by analytic method. A simple and elegant 
proof was given by Pollak, (see [71]), which can be described as follows: Add an 
additional space n and arrange all n+ 1 spaces clockwise in a circle. Again assume 
that n cars enter the street one at a time, each with a preferred space a; € {0,1,...,n}. 
Preference a; = n is treated like any other preference: If space n is occupied, car C; 
moves clockwise to the first unoccupied space. Every sequence of preferences leaves 
one space unoccupied, and because of symmetry the number of sequences leaving a 
given space, say k, unoccupied is the same for every k, 0 < k <n. Hence the number 
with k =n, which is the number of parking functions, is <a of the total number of 

(n + 1)” n—-1 
ery ee (n+1)"™.” 

It is immediately noticed that (n+ 1)"~! is the number of labeled trees on n+ 1 
vertices, by the famous Cayley’s formula [18]. Many bijections between the set of 
parking functions and the set of labeled trees are constructed. They reveal deep con- 
nections between parking functions and other combinatorial structures, and lead to 
various generalizations and applications in different fields, notably in algebra, inter- 
polation theory, probability and statistics, representation theory, and geometry. In this 
chapter we survey the basic results and developments on the combinatorics theory of 
parking functions in the last 20 years. 


preference sequences. This gives 


Notation. We write [n] for the set {1,2,...,n}, and [n]o for the set {0,1,...,n}. 
We will often think of a function f : [n] — |n] as the sequence of its values 
f(1), f(2),...,f(m). If (hj) is a sequence, we will use the boldface letter h to rep- 


“In literature, some people assume that a; are positive integers and use the condition that 1 < aj <i. 
Throughout this chapter we always allow the value of 0 and require that 0 < aj) <i. 
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resent it. The letter a is reserved for a parking function of length n. We denote by 
PAH , the set of parking functions of length n, 

A labeled tree on [n]o is a connected graph on the vertex set [n]o with no cycles. 
It is equivalent to a labeled rooted forest on [n] obtained from the tree by deleting 
vertex 0 and replacing edges connecting 0 to vertex i by a root at i. A labeled rooted 
forest on [n] is also called an acyclic function on [n]. We denote by .%,; the set of 
labeled trees with the vertex set [n]o. 


13.2 Parking functions and labeled trees 


There are many bijections between the set of parking functions of length n and the 
set of labeled trees on [n]o, for example, see [72, 71, 51, 30, 31, 56, 65]. Here we 
introduce some elegant constructions, each of which reveals an intrinsic property of 
parking functions. 


13.2.1 Labeled trees with Priifer code 


A Priifer code, first introduced by Priifer [70] to prove Cayley’s formula, associates 
with each labeled tree with n vertices a unique sequence of length n — 2. Given a 
tree T € %,41, its Priifer code is generated by iteratively removing vertices from 
the tree until only two vertices remain. Specifically, at step i, one removes the leaf 
with the smallest label and sets the ith element of the Priifer sequence to be the label 
of this leaf’s neighbor. For instance, the Priifer code of the tree in Figure 13.1 is 
(2,4,7,0,2,3). 


7 
3 4 
2 5 
1 0 
6 


Figure 13.1 
A labeled tree with Priifer code (2,4,7,0,2,3). 


A bijection between Y.%,, and %,41 was constructed by Pollak [71] using 
Priifer code. 
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Theorem 13.2.1 For each parking function a = (a,,d2,...,@y), define the difference 
sequence (C1,C2,.--,Cn—1) by letting 
Ci=4i41—Q; modn+l. (13.1) 

Let T (a) be the labeled tree in %,4 whose Priifer code is (c1,C2,...,Cn—1). Then the 
map a—+ T (a) is a bijection from PX y to ({n\o)""!. 
Sketch of Proof. The inverse equations of (13.1) can be written as 

aj=a,t+e;+---+c;-1,  modn+1 2<ic<n. (13.2) 


That is, a parking function is determined by its difference sequence if a; is known. 
The following algorithm describes how to find an a; for each (c1,c2,...,Cn—1) € 
{0,1,...,n}"7! such that (a1,a2,...,d,) determined by Equations (13.2) is a parking 
function. 


Algorithm 13.2.2 (Algorithm to determine a,) Given (c1,C2,---;Cn-1) € {0,1, 
dasete rs 


I. Let 
hy a 0, 
h; cyt-::-+e¢-1, modn+l, 2<i<n. 


2. Let r(h) = (r0,..-,Tn) be the specification of h, i.e., rj = card{j : hj; = i}. Let 
R(h) =ro+---r7-j-—lfor0<j<n. 


3. Find d to be the smallest index such that Rqa(h) = min{Rj(h):0< j <n}. Then 
aj =n—d. 


The uniqueness of a; follows from the fact that the number of parking functions 
of length n is equal to the number of Priifer codes of length n — 1. | 


Example 13.2.3 For the Priifer code (2,4,7,0,2,5), we have h = (0,2,6,5,5,7,4) 
and hence r(h) = (1,0,1,0,1,2,1,1). 

The sequence R(h) is given by (0,—1,—1,—2,-—2,-1,—1,—1), where the min- 
imal value of Rj(h) is reached at R3(h) = Ra(h) = —2. The smallest index d is 3. 
Hence we have a, =n—3 =4, which recovers the parking function (4,6,2,1,1,3,0). 


A zero in the Priifer code indicates a pair of consecutive like numbers in the 
parking function. Consequently Theorem 13.2.1 leads to an explicit formula for the 
enumerator of parking functions by the number of pairs of like consecutive numbers. 
Precisely, we have the following corollary. 


Corollary 13.2.4 The equality 


git > ai=ai41}| — Cee ae 
AEP KH py 


holds, where |S| is the cardinality of a set S. 
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13.2.2 Inversions of labeled trees 


Let a = (a),...,@,) be a parking function of length n. Consider again the one-way 
street parking scenario, and assume that the car C; parked at space p;, for 1 <i<n. 
Then p1p2--- Pn is a permutation on the letters {0,1,...,2—1}. Let D(a) be the total 
displacement, i.c. 


pia)= F(a) = (5) Pa 


Then D(a) is the total number of failed trials before all the cars find their parking 
spaces. In the language of hashing in computer algorithms, D(a) represents the num- 
ber of linear probing. See Section 13.3.1 for more discussion. 

Define the displacement-enumerator of parking functions as the polynomial 


BRi= Y PM=gD YP giatet), 
acPXH acPLy 


The degree of P,(q) is (@): This polynomial also enumerates labeled trees by some 
important tree statistics, one of which is the number of inversions. 

Let T € 41. View vertex 0 as the root of T. If {i, 7} is an edge of T and j lies 
on the unique path connecting 0 to i, we say that i is the predecessor of j and j is a 
successor of i. In a rooted tree the degree of a vertex i is the number of its successors. 
A leaf is a non-root vertex with no successor. 

An inversion of the tree T is a pair (i, j) for which i < j and j lies on the unique 
path connecting 0 to i. Let inv(T) denote the number of inversions of T. The inver- 
sion enumerator /,,(q) of labeled trees on 1 + 1 vertices is defined as the polyno- 


mial 
Ch mae Se alae 
TET 


which is also called the inversion enumerator of labeled forests on |]. 

Kreweras [56] studied polynomial systems satisfying a recurrence relation, and 
found several combinatorial interpretations, among which are the polynomials [,(q) 
and P,,(q). In Kreweras paper, parking functions are called suites majeures. 


Theorem 13.2.5 


1. The inversion enumerator I,(q) of labeled trees on n+ vertices satisfies the 
recurrence relation 


h(q) = 1, 


Ing) = YE (') (qi tq ' +--+ 1)k(@)n-i(g) 


2. The displacement enumerator of parking functions satisfies the recurrence re- 
lation 


P(g) = 1, 


Pasig) = ¥ ("Ji bgt t+ DAPla) 
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3. Consequently, 
In(q) = Pal): 


In fact, these polynomials are immediately related to connected graphs. 


Theorem 13.2.6 Let C,(q) be the edge-enumerator of connected graphs. Precisely, 
= Wi (G)|— 
G 


where G ranges over all connected graphs (without loops or multiple edges) onn+ 1 
labeled vertices, and E(G) is the set of edges of G. Then 


In(1 +4) = Pa(l+q) =Ci(q)- 


Theorem 13.2.6 implies that the sum of ae Be a) taken over all parking functions 


of length n, as well as the sum of eee ) taken over all labeled trees on n+ 1 ver- 
tices, are equal to the total number of connected graphs with n+k edges onn+ 1 
labeled vertices. Analysis of these generating functions is essential in characterizing 
the evolution of random graphs, see Janson, Knuth, Luczak and Pittel [48]. 

Using the exponential formula on graphs and connected graphs, one derives the 
following generating function identities [81]. 


Theorem 13.2.7 The following identities hold. 


1. 
Y¥ t,(a)( jy Y P,(q)( yr logy Qe 
q)(q- = q)(q— — = log ) g\— 
n>1 n! n>1 ‘ n! n>0 n! 

2. 
n+l yt 
naa" = yg 1yt% = Ea 
n a7h 
n>0 n>0 nl 2049" Ja nl 


Based on Theorem 13.2.5, Kreweras constructed recursively a bijection that car- 
ries the total displacement D(a) of parking functions to the number of inversions of 
labeled trees. A more direct and elegant construction was given by Knuth [51], which 
we describe next. 


Algorithm 13.2.8 (A Bijection ¢ : PK, + %,+1 such that D(a) = inv(@(a))) Leta= 
(a1,---,€n) € AH, and p; be the space that car C; occupies. Let DP; = pi+l1, hence 
P’ = pP>°°-P’, is a permutation of length n. Let q = q192°** qn be the inverse per- 
mutation of p’, that is, car Cg, is parked at the space i —1. 


I. Construct an auxiliary tree by letting the predecessor of vertex k be the first 
element on the right of k and larger than k in the permutation q\q2---qn; if 
there is no such element, let the predecessor be 0. 
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2. Make a copy of the auxiliary tree. Then relabel the nonzero vertices of the new 
tree by proceeding as follows, in preorder (i.e., any vertex is processed before 
its successors): If the label of the current vertex was k in the auxiliary tree, 
swap its current label with the label that is currently (p;,—a,)th smallest in 
the subtree rooted at the current vertex. The final tree is (a). 


Example 13.2.9 Let a = (4,0,1,0,3,6,4). Then p = pi--:pn =4012365 and 
P =p ps: Pp) =5 123476. Consequently, the inverse of the p' is q = 41-**4n = 
234517 6. Figure 13.2 shows the auxiliary tree and the final tree defined by 
Knuth’s bijection. One checks that D(a) = inv(@(a)) = 3. 


auxiliary tree final tree 


Figure 13.2 
The auxiliary tree and the tree ¢(a) corresponding to a = (4,0, 1,0,3,6,4). 


To reverse the procedure, we can reconstruct the auxiliary tree from a labeled tree 
T € %,4, by proceeding in preorder to swap the label of each vertex with the largest 
label currently in its subtree. Comparing the auxiliary tree and the final tree we could 
obtain the values of Pi, — a, for each 1 <k <n. The permutation q) ---g, can be read 
from the auxiliary tree in postorder, which is recursively defined by 


e Ifatree 7 is null, then the empty list is the postorder listing of T. 
e If 7 comprises a single node, that node itself is the postorder list of T. 


e Otherwise, T has root r and the successors of r are t) >t2 >--- > ty. Let 
T; be the subtree with root t;. The postorder listing of T is the nodes of T; in 
postorder, ..., the nodes of 7; in postorder, all followed by the root r. 


For example, the postorder of the auxiliary tree in Figure 13.2 is 23 45 17 6 (0), 
which is q = q1---qn. Taking the inverse of q, one recovers p/p‘ ---p/,, and hence 
Ui°**ap. 

Knuth’s bijection has another nice property: In the tree @(a), if a vertex with label 
t is labeled k in the auxiliary tree, then there are pz; — a, nodes with smaller labels 
than ¢ in the subtree of @(a) rooted at t. In a labeled tree T, call a vertex v leader if 
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it is the smallest among all the vertices of the subtree of T rooted at b. In particular, 
every leaf is a leader. Denote by lead(T) the number of leaders in a labeled tree T. 
On the other hand, for a parking function a we say a car C; is lucky if p; = q;, that 
is, C; is parked at its preferred space. Let lucky(a) denote the number of lucky cars. 
Then Knuth’s bijection @ satisfies lucky(a) = lead(@(a)). 

The statistics “lucky(a)” for parking functions and “lead(T)” for labeled trees 
were studied by Gessel and Seo [37], and Seo and Shin [73], respectively, where 
they gave an explicit formula for the corresponding enumerators as 


ypucky(a) = y yead(T I (i +( n—-it+t 1) )u). 
i=l 


acPXH, TEAL 


A combinatorial explanation of the relation between the inversion-enumerator of 
trees and the edge-enumerator of connected graphs was given by Gessel and Wang 
[38], who used depth-first search (DFS) to partition the set of connected graphs on 
n+ 1 labeled vertices into disjoint Boolean blocks, each of which is represented by a 
labeled tree. This idea is crucial for establishing the connection between G-parking 
functions and the Tutte polynomial of G, (cf. Section 13.5. ) 

Let G be a connected graph on [n]o. The DFS algorithm is applied to G and 
returns a certain spanning tree T = DFS(G) by the following procedure. We start at 
vertex 0 (which is viewed at the root of the tree), and at each step we go to the greatest 
adjacent unvisited vertex if there is one, otherwise, we backtrack. For example, from 
the graph on the left of Figure. 13.3, we get the spanning tree on the right. 


{ Joy 


— 
DFS 


Figure 13.3 
The spanning tree found by DFS on a connected graph. 


Fix a labeled tree T € .%41. Let Y(T) be the set of connected graphs G for which 
DFS(G) = T. Define a set &(T) of edges not in T whose elements are in one-to-one 
correspondence with the inversions of T. More precisely, with every inversion (j,k), 
(j > k > 0), associate the edge between k and the predecessor of j. For the tree on 
the right of Figure 13.3, the edges of &(T) are indicated by dotted lines. 

Gessel and Wang characterized the set of connected graphs in ¥(T). 


Theorem 13.2.10 &(T) consists of those graphs obtained from T by adjoining an 
arbitrary subset of edges in &(T). 
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An immediate corollary of Theorem 13.2.10 is the equation 


gl#(Gl)—" = (1 egy, 
GEY(T) 


Summing over all trees T € .%,41, we have 


Cn (q) y gain = y gun) = In(1 +4). 


G connected TEA 


In the next subsection we will see that applying the breadth-first search on the 
connected graphs leads to the equation between P,(1+q) and C,(q). 


13.2.3 Graph searching algorithms 


For a sequence a = (a1,...,d,) with 0 < a; <n, its specification is the vector 7(a) = 
(ro,.--;/n—1) Where rj = |{j : aj =i}|. If a is a parking function, then the vector 
7(a) is balanced, that is, 


rot---+trj-i-1 > 0 for 0 <i<n, 
ro +::++Tp-1 = 


In addition, define the order permutation o(a) = 0;0)---0,, of a parking function 
a by letting 


oj = |{j : aj <j, or aj =a; and j < i}|. 


In other words, 0; is the position of the entry a; in the non-decreasing rearrange- 
ment of a. For example, for a = (203 048 1 5 4), the specification is 7(a) = 
(2,1,1,1,4,1,0,0,1), and the order permutation is o(@) = 415269387. Clearly 
a parking function determines its specification and order permutation. Conversely, 
knowing the vector 7(a) and the permutation o(a), we can easily recover a by re- 
placing i in o(a) by the ith smallest term in the list 001"! ---(a— 1)". 

Not every pair of a vector 7 and a permutation o can be the specification and 
the order permutation of a parking function. The vector and the permutation must be 
compatible with each other, that is, the terms in the inverse o~! of o are increasing 


Equivalently, the terms | + a rj,2+ Ee, Fisoe ian r; appear from left to right 
ino. 

Let @;, be the set of all pairs (7,0) with 7 € N” and o a permutation of length n 
compatible with 7. 


Theorem 13.2.11 The map p : a— (7(a),o(a)) is a bijection from PH ;, to Cy. 


On the other hand, there are many ways to construct bijections between @;, and 
Jn+1, the set of labeled trees with the vertex set [nm]. Each such bijection, combined 
with Theorem 13.2.11, gives a bijection between Y.%,, and F411. 
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Theorem 13.2.12 The set G,, is in one-to-one correspondence with F,41. 


Here we introduce a family of bijections between @, and 4,41, each of which is 
determined by a choice function and corresponds to a searching algorithm on trees. 
Generally speaking, a searching algorithm checks the vertices of a tree one-by-one, 
starting with the root 0. At each step, we pick a new vertex that is connected to the 
“checked” vertices. The choice function would tell us which new vertex to pick. 

Let IT be the set of all ordered pairs (F,W) such that F is a tree whose vertex set 
V(F) is a subset of [n]o containing the root 0, and 0 4 W C Leaf(F’) where Leaf(F) 
is the set of leaves of F. A choice function y is a function from TI to [n] such that 
y(F,W) € W. Some choice functions are described in Examples 13.2.14-13.2.17. 

Fix a choice function y. Given a tree T € %,41, we define a linear order of the 
vertices V(Y) = v0,V1,V2,---,Vn- First, set vo = 0. For each 0 < i <n, assuming 
Vo,---,Vj-1 are determined. Let W; = {v : the predecessor of v is in {vo,...,vi-1} }, 
and F; be the subtree obtained from T by restricting to W; U {vo,...,vj-1}. Then let 
vi = Y(Fi, Wi). 

For each v;, order the successors of v; from small to large. Let 2) be obtained by 
reading the successors of vg, followed by successors of v;, then successors of v2, and 
so on. Finally, let 7 = (ro,1,---;%n—1) where r; is the number of successors of the 
vertex vj. 


Theorem 13.2.13 The map $y: T — (7y, ity) described above is a bijection from 


F,44 onto G,, where 1, ! 


y 1s the inverse of Ty. 


We explain the bijections with some explicit examples of choice functions, and 
apply each bijection to the tree T in Figure 13.4. 


Figure 13.4 
A tree in .Z0. 


Example 13.2.14 (The vertex-adding order) Let y(F,W) be the vertex in W with 
minimal label. Then the vertex ordering is V(y) =04536728 19. Hence 


Fy = (3,1,2,0,0,1,1,1,0), ay = 467538291. 
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Example 13.2.15 (Depth-first search order) Also known as the preorder traversal, 
the depth-first order is the list of vertices in the order that they were first visited by 
the DFS. Here we adapt the version of DFS described in the end of Section 13.2.2. 

The choice function Yar is defined as Ya¢(F,W) = v where v is the minimal ele- 
ment of W under the depth-first search order. Then the vertex ordering defined by Yur 
is V(df) =0729645 8 1 3. Hence 


Fy = (3,1,1,0,0,1,2,1,0), Ty, = 467295381. 


Example 13.2.16 (Breadth-first search order) Breadth-first search (BFS) is an- 
other commonly used graph searching algorithm, which begins at the root and ex- 
plores all the neighboring nodes before going to the next node. The order that the 
vertices are visited under the BSF is called the BFS order, and denoted by <pr. Ina 
labeled tree T € %,41, let level(i) be the distance of a node i to the root 0. A simple 
version of the BFS order of the vertices of T is to let i <p ¢ j if level(i) < level(j), or 
if level(i) = level(j) but i < j. 

The choice function Yp¢ is defined as Yp¢(F,W) =v where v is the minimal element 
of W under the breadth-first search order <p z. Then for the tree in Figure 13.4, the 
vertex ordering defined by Yr is V(Yap) =0 46725389 1. Hence 


7, = (3,1,0,1,1,2,0,1,0), my, = 467529381. 


Example 13.2.17 (Breadth-first search order with a queue) This is a variation of 
the BFS order in Example 13.2.16. It corresponds to an implementation of BFS with 
a queue structure. It is a vertex ordering, denoted by <pfg, which can be defined 
explicitly by letting i <pyq j if (1) level(i) < level(j), or (2) level(i) = level(j) and 
pre(i) <pfq pre(j), where pre(i) is the predecessor of i; or (3) pre(i) = pre(j) and 
i<j. 

The choice function Ypfqg(F,W) always chooses the minimal element of W under 
the order <p fg. Under this choice function the vertex ordering is0 46752389 1. 
And 

Fogg = (3,1,0,1,2,1,0,1,0), Typ = 467523891. 


Note that in this case, the permutation Ty, ,, is the same as the vertex ordering (after 
removing 0). 


Theorem 13.2.11 and the bijections given in Theorem 13.2.13 have some inter- 
esting implications. For example, combining the map p of Theorem 13.2.11 and any 
bijection of Theorem 13.2.13, we get that the number of parking functions with k 
entries equal to 0 is equal to the number of labeled trees on n+ 1 vertices whose 
root vertex has degree k, which in turn equals the number of rooted forests on [n] 
with k tree components. The latter is well-known to be Cae . Written in terms 
of generating functions, we have 


Corollary 13.2.18 The enumerator of parking functions by the number of elements 
equal to 0 is x(x+n)"|, i.e., 


xt: =O} — x(x en)r!, 
ac PH 
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Next we describe a special implementation of the breadth-first search algorithm 
with a queue structure, which finds a particular spanning tree for any connected 
graph. The algorithm, combining with Theorems 13.2.11 and 13.2.13, gives the com- 
binatorial explanation of the equation P,(1 +4) =C,(q). 

Let G be a connected graph on [n]o. The BFS algorithm is described as a queue 
Q that starts at vertex 0. At each stage we take the vertex x at the head of Q, remove 
x from Q, and add all unvisited neighbors y of x to Q in numerical order. We will 
call that operation “processing x.” We continue the above procedure until the queue 
is empty. The spanning tree T = BF S(G) is obtained by adding all edges of the form 
{x,y}, where x is the vertex being processed, and y is an unvisited neighbor of x. For 
the connected graph G in Figure 13.3, the BFS gives the spanning tree shown on the 
right-hand side of Figure 13.5. 


ea 


eu 
Ge 


N 


Figure 13.5 
Spanning tree of Figure 13.3 found by BFS, where the dotted edges belong to 6 (T). 


Let r; be the number of vertices found by the rth vertex processed, fort =0,...,n, 
which in our example, are (70,71,.--,76) = (2,1,1,0,2,0,0). Note that r, is always 
0. Let gq, be the size of the queue after the ¢th vertex is processed. Then go = | and 
dt = GU-1+%~1 — 1, which, in our example, are 1,2, 2,2, 1,2, 1,0. 

For the example in Figure 13.5, the queue Q, 7; and q; at each stage ¢ are listed in 
the following table. 


t 0 1 2 3 4 5) 6 7 
Q 0 3 4 1 5 2 6 0 
4 1 5 6 

rt 2 1 1 0 2 0 0 
at 1 2 2 2 1 2 1 0 


A necessary and sufficient condition for the graph being connected is that the 
sequence {q;} satisfies 


Qn+1 = 9, and qi >0 fori<n. 
In terms of 7;, the above constraints are equivalent to 


rot--:rj-1 >ifori<n, rot-:-+m_1 =n. 
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In other words, 7 = (ro,..-,%n—1) is balanced. Also note that the queue uniquely 
determines a permutation 7 of length n, namely, the order that the vertices are pro- 
cessed in the queue. One can read it from the head of the queue. In our example, it 
is 341526. The vector 7 and the permutation z~! are compatible. Actually, they are 
uniquely determined by the spanning tree T found by the BFS. 

Let a(T) be the parking function that corresponds to the pair (7,2~') by the 
bijection p of Theorem 13.2.11, where T = BFS(G). One easily computes that 
the displacement of a(T) is D(a(T)) = (3) — Lip ii. In our example, a(G) = 
(1,4,0,0,2,4), and D(a(G)) = 4. 

Let Y(T ) be the set of connected graphs for which the spanning tree found by the 
BFS is T. The following crucial observation is due to Spencer [76]: An edge {i, j} 
can be added to T without changing the spanning tree under the BFS if and only if 
in the queue, when the first of the two vertices is processed, the other is currently in 
the queue. Let &)(T) be the set of all such edges. In our example, é\(T) = {{3,4}, 
{4,1}, {1,5}, {2,6}}. See Figure 13.5 to compare with the table of the queue. 

It follows that 


Theorem 13.2.19 Y(T) consists of these graphs obtained from T by adjoining an 
arbitrary subset of edges in &\(T). 


Thus 


y gl Ol" = (14 4AM, 
GEG (T) 


Now we compute |é)(T)|. From the queue Q, we have 
n n n—1 nat 
ar = Fe = Plot tni-9= Foon (4). 
i=1 i=l i-0 


Since ar rj =n, the number of the last formula equals 


Si bin (“S\-(0) -Fin= paar). 


Thus 
y gll(@l- =(1 peg yO). 
GEY,(T) 


Summing over all trees T on |n]o, we obtain 


Chaya Deg nay egy eh Blea): 


G connected TET A 


13.2.4 External activity of labeled trees 


Another combinatorial statistic of labeled trees that corresponds to the displacement 
of parking functions is the external activity, a notion proposed by Tutte in defin- 
ing a bivariate polynomial, called Tutte polynomial, for undirected graphs. Tutte 
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polynomial plays an important role in graph enumeration. The evaluations of Tutte 
polynomial at various points give the numbers of spanning trees, spanning forests, 
connected subgraphs, acyclic orientations, etc. Here we just recall the definition of 
external activity. In Section 13.5.3 we will present a much stronger relation between 
Tutte polynomial and general parking functions. 

Consider the complete graph K on [n]o. Fix a total ordering of its edges. For a 
tree T € F141, an edge e € K —T is externally active if it is the smallest edge in 
the unique cycle contained in T U {e}. The external activity ea(T) is the number 
of externally active edges of T. Define the external activity enumerator of trees in 


Tn4\ as 
FAg= yf. 
TET 
Then 
EA,(q) =In(q) = Pr(): 

The equation EA,(q) = 1I,(q) was proved by Bjérner [13] using his results on 
shellability and homology in matroids. Beissinger [9] constructed a bijection from 
parking functions to labeled trees that carries the displacement of parking functions 
to the external activity of trees. 


13.2.5 Sparse connected graphs 


Theorem 13.2.6 has a nice application in graphical enumeration. Let c(n + 1,k) be 
the number of labeled connected graphs on n+ 1 vertices with exactly n+k edges. 
For example, c(n + 1,0) = (n+1)""! by Cayley’s formula. Theorem 13.2.6 implies 


that ; 
J 
c(n+1,k) -Doi(i). 
J 


where p; is the number of parking functions that have displacement D(a) = j. Let 
F,.(n) be the kth falling factorial moment of D(a), ie., 


Al)= aoe YO k= poe YH (%), 


(M1) oy, (n +1)" Fx, 
where (n), is the falling factorial n(n — 1)---(n —k+ 1). It follows that 


Theorem 13.2.20 


n n—1 
c(n+1,k) = Fun) 


Theorem 13.2.20 can also be obtained by using a result of Spencer [76] that 
c(n+1,k) 3 M 
c(n+1,0) k 


where M is a certain random variable defined on all labeled trees with n + 1 vertices 
with uniform distribution. This formula, together with the bijection between labeled 
trees and parking functions, yields Theorem 13.2.20. 
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For k = 1,2, the formulas for F,(n) are computed in [36, 52] as 


nm 8 astes 


i=2 
F,(n) = TE (15a 1.n—3)+702(0+ 1 3) +201 (n+ 10-3), 
where 
O,(mn)=¥ ee ie a aba) (13.3) 
k>0 


Explicit formulas for higher moments are computed by Kung and Yan [58, Theorem 
7.1] by solving a recursion based on a combinatorial decomposition of parking func- 
tions. The decomposition is the key idea in connecting parking functions to Gonéarov 
polynomials, a system of polynomials from interpolation theory that provides a nat- 
ural tool for studying the algebraic properties of parking functions. 
In [84] Wright found the following asymptotic formula: for fixed k and n tending 
to infinity, 
c(n+1,k) = pri(n +1)" 2 (1 4.0(n7 1/7). 


The Wright constants p, are given by 


gp} /29(1-3k)/2g, 


P= TGR)” 


where 0; is defined by a second order recursion 


Ox41 = 


lame 


-1 
OsOx-s (kK >2), 
2 s=1 


with initial values 69 = 1/4, 0; = 5/16, and 02 = 15/16. The first several values of 
Px are po = V27/4, pi = 5/24, and po = 5/27/28. Using combinatorial analysis, 
Kung and Yan [58] gave another formula for the Wright constants p;, which only 
involves a linear recurrence. 


Remark 13.2.21 Theorems 13.2.11 and 13.2.12 are due to Foata and Riordan [30], 
where they used the breadth-first search order of Example 13.2.17 to prove Theorem 
13.2.12. A simpler version of choice functions first appeared in Francon [31] as “se- 
lection procedures” that lead to a family of bijections between parking functions and 
labeled trees, each corresponding to a searching algorithm. The idea was extended 
to parking functions associated with graphs by Chebikin and Pylyavskyy [21], and 
Kostic and Yan [54]. Spencer [76] used the implementation of BFS with a queue to 
develop an exact formula for the number of labeled connected graphs on [n] with 
n—1+k edges (k fixed) in terms of appropriate expectations. Moving to asymp- 
totics, Spencer showed that the expectations can be expressed in terms of a certain 
restricted Brownian motion. Spencer’s approach was generalized by Kostic and Yan 
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[54] to establish the relation between parking functions associated to a graph G and 
the Tutte polynomial of G, (cf. Section 13.5). Results of [54] also reveal the connec- 
tions between external activities of the spanning trees of a graph G and the parking 
functions associated to G. 


13.3. Many faces of parking functions 


The parking function is an object lying in the center of combinatorics and appearing 
in many discrete and algebraic structures. In this section we describe some of the 
most important examples and their implications. 


13.3.1 Hashing and linear probing 


An efficient method for storing and retrieving data in computer programming is 
known as hashing or scatter storage technique. It has a hash function / that as- 
signs a hash value A(K) to each item K. However, two or more items may have the 
same hash value and hence cause a hash collision. Linear probing and insertion is 
a simple and basic algorithm for resolving hash collisions by sequentially searching 
the hash table for a free location. It inserts n items in m > n cells by the following 
rules. Begin with all the cells (0,1,...,m— 1) empty. Then, for 1 < k <n, insert the 
kth item into the first nonempty cell in the sequence My, + 1,4, +2,...(mod m), 
where /1; is the hash value of the kth item and is in the range 0 < y < m. For acom- 
prehensive description of hashing, as well as other storage and retrieval methods, see 
Section 6.4 of [51]. 

In the above description, the sequence (/1,...,4,) is called the hash function. 
It is confined if the linear probing with /),...,h, will leave the cell m— 1 empty. 
One notes immediately that when m =n + 1, the confined hash functions are exactly 
parking functions of length n. 

If the kth item is inserted into position px, then D(h) = Y7_, [(pe — te) ~mod m] 
is the number of linear probes, or the total displacement of the items from their 
hash addresses, which gives the name displacement of the corresponding statistic of 
parking functions. Let Dnn(q) = Xn gP(h), where h ranges over all m” possible hash 
functions (h1,...,/n) with 0 < hy < m, and let Finn(q) be the same sum restricted 
to confined hash functions. Using Pollak’s argument with m cells, one notes that 
given (h1,...,4,), the m hash sequences {(4;+ j) modm,0 < j < m} all have the 
same total displacement, and exactly (m—n)/m of them are confined. Therefore 
Finn(Q) = “=="Dymn(q), and the probability generating function for D(h) satisfies 


m 


eee Apa (13.4) 
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Equation (13.4) allows us to reduce the computation of the probability distribu- 
tion of D(h) over random hash functions to that of confined hash functions, which 
is easier since the linear probing does not “wrap around” when the hash function 
is confined. In particular, we have F,+1.(q) = P,(q), the displacement enumerator 
of parking functions of length n. In general, for m =n +r, a confined hash function 
h=h,,...,h, leaves the cells ny <nz <-+- <n, empty if and only if h is obtained by 
merging r parking functions of lengths n,},n2 —n, —1,...,n,;—n,;_; —1. Lets; =n 
and sj; =nj—nj—; — 1 for2 <i<r. Then Y)_,s; =n, and 


n 
Frtrn(q) = y « a laR (q)---P;,(q). (13.5) 
sp +so+e-+5-=n 1,52,-++55r 


Let 


It follows from Equation (13.5) that 


n 


% 
Y Fr+rn(9) Cole 


n>0 


In other words, the distribution of total displacement for linear probing with random 
hash functions is determined by the exponential generating function of P,(q) for 
parking functions. Some identities related to F(g,z) and its variations are given in 
Theorem 13.2.7. 

In analyzing the performance of hashing as a storage method, one is interested 
in the expected value of D(h) over all hash functions h : [n] — |[m], assuming all 
are equally likely. This value is D’,,(1)/Dmn(1), which equals Fy, ,(1)/Finn. Us- 
ing combinatorial analysis and the Lagrange inversion formula, one gets an explicit 
formula for the expected value of D(h), (see e.g. [36, 52]), 


Theorem 13.3.1 The expected value of linear probes D(h) as h varies over all hash 
functions from {n] to [m] (n < m) is 


where Q,(m,n) is defined in (13.3). 


Knuth also computed the second factorial moments of D(h), and obtained the fol- 
lowing expected value of D(h)(D(h) — 1) in Formula (5.5) of [52]. 


Exp|D(h)(D(h) — 1)] 
= mn P=) sos(m.n— 3) + (443m — 3n)Q2(m,n — 3) 
+(5—3m-+3n)Q1(m,n—3)]. 


Moment analysis and characterizations of limit distributions for the linear probes 
are carried out by Flajolet, Poblete and Viola [29], and Janson [47]. 
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13.3.2 Lattice of noncrossing partitions 


A set partition of {n] is a family of pairwise disjoint nonempty subsets By,...,Bx 
whose union is [n]. A partition {B,,...,B,} is noncrossing if there are no elements 
a<b<c<dsuch that a,c € B;, b,d € Bj andi ¢ j. The study of noncrossing parti- 
tions can be traced back to Becker [8] under the name “planar rhyme schemes.” The 
systematic study of noncrossing partitions began with Kreweras [55] and Poupart 
[69], and became one of the central topics in contemporary combinatorics. 

Let z and 7’ be two partitions of [n]. We say that z is a refinement of 7’ if ev- 
ery block of 7 is contained in a block of 2’. Refinement induces a partial order on 
the set of partitions: z < z’ if and only if 7 is a refinement of 2’. Under this order 
there is a unique maximal element, the partition with only one block [n]. In addition 
any two partitions z and 7’ have a greatest lower bound z A 2’ whose blocks are the 
non-empty intersections of one block of z and one block of 2’. It is easy to check 
that if both z and z’ are noncrossing, then so is z \ 2’. Hence the set of noncrossing 
partitions form a lattice, which is denoted NC, The lattice of noncrossing partitions 
enjoys a number of remarkable properties, and plays a fundamental role in the com- 
binatorics of Coxeter groups and free probability. See [1] and [66] for references. 
Next is a list of basic properties of lattice of non-crossing partitions. 


1. It is well-known that the number of noncrossing partitions of [n] is the nth 


Catalan number C,, = aa ea 


2. The lattice NC, is graded of rank n — 1. The rank of a noncrossing partition 7 
is n — |z|, where |z| is the number of blocks in z. 


3. The number of noncrossing partitions with exactly k blocks is given by the 
Narayana number N(n,k), where 


won -20) (9) 


4. The lattice NC,, has the minimum 0 = {1}{2}--- {n} and the maximum 1 = 
[n}. 


5. NC, is self-dual, i.e., there is an isomorphism & : NC, — NC, such that 7 < o 
if and only if €(o) < €(z). The Kreweras complement operation provides 
such an isomorphism, [55]. 


A surprising connection between parking functions and the lattice of noncrossing 
partitions was revealed by Stanley [80], who constructed an edge labeling of NC,,+1 
whose maximal chains are labeled by the set Y.%», The labeling also leads to a 
local action of the symmetric group G, on NC,,+1. 

In a locally finite poset P, an edge of P is a pair (u,v) such that v covers u, i.e., 
u <vand there is no element t satisfying u <t¢ < v. For NC,+1, a partition 2’ covers 
partition z if and only if z’ is obtained from z by merging two blocks into one. Let 
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(2,7) be an edge of NC,,,; where two blocks B and B’ of z are merged to form one 
block of 2’. Suppose that min B < min B’. Define 
A(z,7') = max{ie B : i< minB’}. 


For instance, if B = {2,3,6, 16} and B’ = {8,9, 10,14}, then A(z, 2’) = 6. This la- 
beling is well-defined since the set {i€ B : i< minB’} is nonempty. 
Figure 13.6 shows the labeling A on the lattice NC3. 


tae} 
2 1 | 
{1,2},{3} {1,3}, {2} {1}, {2,3} 
I 1 2 
{1}, {2}, {3} 


Figure 13.6 
The labeling A on NC3. 


The labeling A of the edges of NC,,+1 extends in a natural way to a puss of 
the maximal chains. Namely, given a maximal chain m: 0= MH <M << Ay = i 
where 7; covers 7;_1, we set 


A(m) = (A(Z0, 7), A(T, 7), as A(Tn—1, Tn))- 
Note that A(7;, 741) € [n]. Adjust the values by letting 
Ai (m) — (A(%0, 71) Ke 1,A(71, 72) ng 1, Cate A(Tn-1; Tn) — 1). 


Theorem 13.3.2 The label A,(m) of the maximal chains of NCj+1 consists of the 
parking functions of length n, each occurring once. 


As an immediate consequence, we recover a result of Kreweras [55]. 


Corollary 13.3.3 The number of maximal chains of noncrossing partitions of 
{1,2,...,2+1} is (n+1)"7). 

The labeling A induces an R-labeling by letting A*(z,o) = |x| — A(a,o), as in 
the sense of [78, Def. 3.13.1]. More precisely, for every interval [7,7’] of NCn+1, 
there is a unique maximal chain m: 7 = 1 <M <-+-<aj= x’ such that 


A*(m0,%1) < A* (M72) <2 < A* (1j-1,%;). 


854 Handbook of Enumerative Combinatorics 


For any poset equipped with an R-labeling, there is a general theorem [78, Theorem 
3.13.2] that allows us to enumerate the labeling of maximal chains with a given 
descent set in terms of the rank-selected Mobius invariants. Applying this theorem to 
NC,+1 and parking functions yields the following result. 

For a finite graded poset P of rank n with 0 and j and with rank function p, let S 
be a subset of [n — 1] and denote a(S) to be the number of chains 0 = to <ty <-+- < 
t, = 1 of P such that {p(t;), P(t2),...,P(t,1)} =S. The function a(S) is called the 
flag f-vector of P. Further define 


Be(S) = (-1)! Map (7), 


TCS 


or equivalently, 
ap(S) = )! Br(T). 
TCS 
The function Bp is called the flag h-vector of P. 
For a parking function a = (a),...,a,), define the descent set Des(a) by letting 


Des(a) = {i : aj; > aj+1}. 
Theorem 13.3.4 Let SC [n— 1]. 


1. The number of parking functions a of length n with Des(a) = S is equal to 
Brey ([2 — 1] — 5). 
2. The number of parking functions a of length n satisfying S C Des(a) is equal 
to ONnc,,,,({2— 1] —S). 
The value of Qnc,,,(7) for T C [n—1] is computed in Theorem 3.2 of [26]. 
Assume T = {t) <to <-:: <t,}. Set to = 0, t-41 =n, and 6; =t;—t;-1 for 1 <i< 


r+1. Then 
yo n+1 
anc, (T) = -[] ( 5; i: 


Niy 


Theorem 13.3.2 is used by Kim and Seo [50] to study the minimal transitive fac- 
torizations for permutations of cycle type (n) and (1,n — 1). Given a permutation o 
in G,, the minimal transitive factorizations of o is a set of n+ ¢—2 transpositions 
that generate the full symmetric group such that their product is o, where @ is the 
number of cycles of o. If the permutation o is of cycle type A = (Ay,...,A¢), then 
the number of minimal transitive factorizations of © is 

0—2)! -3 t aj 
(n+ \in Nasr 


a formula originally suggested by Hurwitz but proved by Goulden and Jackson [41], 
and Bousquet-Mélou and Schaeffer [16]. In particular, when o is of cycle type (n) 
or (1,2 —1), Hurwitz’s formula reduces to n"~* and (n—1)". Kim and Seo [50] 
presented combinatorial proofs for these two cases using an interplay between the 
minimal transitive factorizations, cycle chord diagrams, Stanley’s labeling of maxi- 
mal chains of NC, and parking functions. 
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13.3.3. Hyperplane arrangements 


A (real) hyperplane arrangement is a discrete set of affine hyperplanes in R”. An 
especially important arrangement is the braid arrangement %,,, which consists of 
all the hyperplanes x;— x; =0 for 1 <i< j <n. Fora hyperplane arrangement /, if 
we remove the union of the hyperplanes from R”, then we obtain a disjoint union of 
open cells, called the regions of ./. Fix a region Ro of and call it the base region. 
Given a region R of &, let d(R), the distance of R, be the number of hyperplanes 
H in & that separate Ro from R, i.e., Ro and R lie in different sides of H. Define the 
distance enumerator of ./ (with respect to Ro) to be the generating function 


Dg(q) = yf, 
R 


where R ranges over all regions of .&/. For finite </, D.y(q) is a polynomial in g, and 
D.,(1) is the number of regions of the hyperplane arrangement ./. 

As an example, consider the braid arrangement &, that has n! regions, each 
corresponding to a permutation of G,. It is natural to let Ro be the region defined 
by the conditions x; > x2 > --- > X,. There is an elegant way of labeling the regions 
of Z, by integer sequences that leads to the formula of Dg (q). Let e; € N” be the 
vector with a | in the ith coordinate and 0’s elsewhere. First label the base region 
Ro by A(Ro) = (0,...,0). Suppose now that R has been labeled, and that R’ is an 
unlabeled region that is separated from R by a unique hyperplane x; = x;, where 
i< j. Then set 

A(R’) = A(R) +4. 


It is easy to check that this labeling is well-defined, independent of the order in which 
the regions are labeled, and the labels of regions of &, are the sequences (c1,...,Cn) 
such that 0 < c; <n—i. Hence 


Da,(q) =(1+q)(1+q+@)--(l+q+--+4"1), 


the standard g-analog of n!. 

A deformation of the braid arrangement is the Shi arrangement .%, consisting of 
hyperplanes x; —x; =0,1 for 1 <i < j <n. This deformation was first considered by 
Shi [74] in his investigation of the affine Weyl group A,,. Shi showed by grouptheo- 
retic techniques that the number of regions of Shi arrangement is (n + 1)"~!. Pak and 
Stanley [79] gave a bijective proof by constructing a labeling of .%, that is analogous 
to that of F,,. 


Algorithm 13.3.5 (A labeling of .%,) Let the base region Ro be defined by x, > x2 > 
++ > XxX, and Xx; — Xn <1. Note that this is the unique region of %,, contained between 
all pairs of parallel hyperplanes. Label Ro by the n-tuple (0,0,...,0). Now suppose 
a region R is labeled, and R' is an unlabeled region that is separated from R by a 
hyperplane H of -%,. Then define 


A(R’) = A(R) +e, if H is given by xj—xj =Owithi< j 
| A(R)+e;, if His given by x; -—xj =1 withi < j. 
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The labeling A is independent of the order of the hyperplanes, hence well-defined. 
In addition, if A(R) = (a1,...,@n), then d(R) = ay +--+ +p, that is, there are a, + 
-++ +a, hyperplanes of -Y, that separate R and Ro. 


Figure 13.7 shows the labeling of .“4, where the hyperplanes are projected to 
Xp $X2 +23 =0. 


2-3 = 1 XxX) —x3=0 


xyy—x3= 1 xy -23=0 


Figure 13.7 
The labeling A(R) for the Shi arrangement .%3. 


Theorem 13.3.6 The labeling 4 defined above is a bijection from the regions of Sy 
to the set PX ,, of all parking functions of length n. Consequently, the number of 
regions R for which i hyperplanes separate R from Ro is equal to the number of 
parking functions of length n with displacement D(a) = (5) —i. 


In addition to Theorem 13.3.6, Stanley [79] generalized the labeling of .%, to 
the extended Shi arrangement, and established connections between the extended 
Shi arrangement and other combinatorial subjects, in particular, generalized parking 
functions and rooted k-forests, (c.f. 13.4.4). 
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Athanasiadis and Linusson [5] constructed another simple bijection between the 
regions of the Shi arrangement and the set of parking functions. Their bijection can 
be generalized to any subarrangement of .~, containing the hyperplanes x; — xj; = 0 
and to the extended Shi arrangements. It also implies that the number of relatively 
bounded regions of .%, is (n— 1)"~!, where a region is relatively bounded if its 
intersection with the hyperplane x; +---+.x, =0 is bounded as a subset of Euclidean 
space. Athanasiadis and Linusson’s bijection maps the relatively bounded regions of 
-Y, to the prime parking functions of length n, a concept due to Gessel. Under our 
notation a prime parking function of length n is a sequence (a1,...,a,) of non- 
negative integers such that for all 1 < j <n—1, the cardinality of the set {a;: a; < j} 
is at least 7+ 1. 

Armstrong [2] considered the regions of the Shi arrangement as antichains or 
ideals in the poset of positive roots, and mapped the regions bijectively to certain 
labeled Dyck paths. The same labeled Dyck paths are commonly used in the study of 
diagonal harmonics to encode parking functions, (c.f. Section 13.6). Combining these 
two results one gets essentially the same bijection of Athanasiadis and Linusson’s. 
There is some further elaboration of this labeling scheme in [4], which reveals the 
relations between the Shi arrangement and other hyperplane arrangements, as well 
as the relations among various statistics of parking functions. 


13.3.4 Allowable input-output pairs in a priority queue 


A priority queue is an abstract data type equipped with the operations INSERT and 
DELETEMIN. Let 6 = 010)...0, be a permutation of [n]. Each INSERT operation 
will insert the next element of o into the queue, and each DELETEMIN operation 
will remove the minimal element of the queue and place it in the output stream. A 
sequence of m INSERT and n DELETEMIN is an allowable sequence if any initial 
subsequence contains at least as many INSERT’s as DELETEMIN’s. The application 
of an allowable sequence to a permutation o is called a priority queue computation. 
Assume the outcome is T. Then (6,7) is called an allowable pair of permutations in 
a priority queue. 

For example, when n = 3, the allowable sequences are IDIDID, IDITDD, 
IIDIDD, ITDDID, IITIDDD, where I stands for INSERT and D for DELETEMIN. 
There are 16 allowable pairs of permutations, as listed below. 


(123, 123) | (132, 132) | (132, 123) | (213, 213) 
(213, 123) | (231, 231) | (231, 213) | (231, 123) 
(312, 312) | (312, 123) | (312, 132) | (321, 321) 
(321, 312) | (321, 213) | (321, 231) | (321, 123) 


Let 2, be the set of all allowable pairs of permutations on [n]. Atkinson and 
Thiyagarajah [6] found that the number of allowable pairs in 2, is (n+1)"~!. Gilbey 
and Kalikow [39] constructed a bijection between 2, and Y.% ,, that for each allow- 
able pair (0,7), the corresponding parking function has t as output, that is, the car 
C,, is parked at the space i— 1 under the parking rules. In other words, T is the per- 
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mutation q as defined in Knuth’s bijection @ described in Section 13.2.2. In addition, 
the bijection of Gilbey and Kalikow’s has the extra property of preserving the set of 
breakpoints. For a parking function a = (a),...,d,), an integer b € {0,1,...,n}isa 
breakpoint if and only if |{i: p(i) < b—1| = b, while for an allowable pair (0,7), 
b € [0,n] is a breakpoint if and only if {o1,...,0,} = {t1,...,%}. Note that under 
these definitions, 0 and n are always breakpoints of ac A.%,, and any (0,7) € Dy. 


13.3.5 Two variations of parking functions 


We conclude this section with two variations of parking functions. 

The first variation is the defective parking functions considered by Cameron, Jo- 
hannsen, Prellberg, and Schweitzer [17]. Suppose that there are m drivers entering 
a one-way street with n parking spaces, each with a preferred parking space. The 
parking procedure is the same as before. If k drivers fail to park, the preference se- 
quence is called a defective parking function of defect k, whose number is denoted 
by cp(n,m,k). The original parking functions correspond to the case that m = n and 
k=0. 

For the case m <n and k = 0, Pollak’s argument can be adapted to give 
cp(n,m,0) = (n+ 1—m)(n+1)""!. The formula for general case is much harder. 
Cameron et al. established a recurrence relation for cp(n,m,k), and expressed it as 
an equation for a three-variable generating function. Solving the equation by using 
the kernel method, they obtained the following result. 


Theorem 13.3.7 The number of defective parking functions of defect k is given by 
cp(n,m,k) = S(n,m,k) —S(n,m,k +1), 


where S(n,m,k) is the number of car parking preferences of m cars on n spaces, 
such that at least k cars do not find a parking space. The values of S(n,m,k) can be 
computed explicitly by 


if a™ ifk<m—n, 
Se a { Sa (")(n—m+k)(n—m+k+i)|\(m—k—i)™" otherwise. 
These formulas were also derived by Pitman and Stanley [67] using the volume poly- 
nomial of a polytope, and by Yan [86] with combinatorial means. 

Cameron et al. investigated the asymptotic behavior of defective parking func- 
tions. They proved that if m = n+ |y,/n], then the limiting probability of at most 
|x,/n| drivers failing to park is 


1 Bva l-e *™)  ifx>y 
Urs d cp(n,m,k) = { 0 otherwise. 


This limit distribution implies that if m <n-+k for a fixed constant k, then 


cp(n,m,k) 2 


Pea 
nm n 


(2k m+n) ek kn) /n 
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Another problem for defective parking functions is to find the limiting probability 
that all parking spaces are occupied. For m < n this probability is clearly 0. It turns 
out that when m is linear in n, that is, m = |cn| for a constant c > 0 andk =m-—n, 


cp(n,m,k) 0 ife <1, 


aj [pC \i— 13.6 
poe. py = rest: ee) 


Formula (13.6) is proved in Theorem 10 of [17] and also in [77] by Spencer and Yan 
who used the Galton-Watson branching process to investigate the connections be- 
tween parking functions and random labeled trees. The approach of [77] was further 
extended by Chassaing and Marckert [20] to obtain tight bounds for the moments of 
the width of rooted labeled trees. 

The second variation was given by Zara [88] who described an interesting parking 
problem. Assume that in the middle of the well-known parking procedure, n —k 
spaces are already taken, and the spaces q = {q1 < q2 <--: < qx} are still available. 
Some other k cars want to take these spaces; they enter the street and start advancing. 
When they are in front of spaces p = {pi < p2 <--: < px}, the lights go off. The 
cars can only advance towards the end of the street and park in an unoccupied space. 
Will all of them be able to find parking spaces? 

It is clear that all the cars can park if and only if p; < q; for all 1 <i<k. If this 
condition is satisfied, we say that p < q and call p,q an initial condition. Given an 
initial condition p,q, a possible final arrangement is given by a permutation T € Gx: 
for each i = 1,...,k, the car at p; takes the space q,,;). A permutation T is attainable 
from p,q if tT can appear as the permutation associated with a final arrangement 
when the initial condition is p,q. In particular, if everyone plays safe and takes the 
first available space, just as what is defined in the notion of parking functions, the 
resulting T is called the safe permutation for p,q. 

Zara gave a complete characterization of safe permutations: If Tt is a safe permu- 
tation, then it avoids pattern 231. Conversely, a 231-avoiding permutation Tt can be 
realized as the safe permutation for some initial conditions if and only if is(t) is at 
least 2k — n, where is(T) is the maximal length of the increasing subsequences of T. 
There is a surprising connection between the safe permutations and the paths in the 
Johnson graph J(n,k), which is the graph whose vertices are the k-element subsets 
of [n], and whose edges are (V;,V2) such that V; V2 has size k— 1. 


13.4 Generalized parking functions 
13.4.1 u-parking functions 


For a finite sequence (x1,x2,...,%,) of real numbers, rearrange the terms in non- 
decreasing order as x(}) < x(2) < +++ < x(q). The term x;;) is the ith order statistic 
of the sequence x. Let u be a non-decreasing sequence (u1,u2,U3,...) of positive 
integers. A u-parking function of length n is a sequence (x1,x2,...,%») satisfying 
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0 < x(j) < uj. The parking functions studied in previous sections correspond to the 
case u = (1,2,...,7), and are usually referred as ordinary parking functions in enu- 
meration literature, or as classical parking functions in diagonal harmonic research 
(c.f. Section 13.6). However, in this paper we shall reserve the words “classical park- 
ing functions” to the case that u is an arithmetic progression (c.f. Section 13.4.4). 

Let Y.% ,(u) be the set of u-parking functions of length n, and PK,,(u) the num- 
ber of elements in Y.% ,,(u). Clearly PK,,(u) is a function of u1,...,U,. Less obvious 
is the fact that it is a homogeneous polynomial of uw, u2,...,u,. Furthermore, there 
is a determinant formula for PK, (u). 


Theorem 13.4.1 The number PK,(u) of u-parking functions of length n equals 
n\det(D), where D is the n x n matrix with ijth entry equal to 


i j4] 
u! i+ 


(Gj-i+1)V’ 
if j-—i+1 > 0 and 0 otherwise. 


Theorem 13.4.1 is the discrete analog of a formula of Steck [82] for the cumulative 
distribution function of the random vector for order statistics of n independent ran- 
dom variables with uniform distribution on an interval. However, Steck’s formula is 
for sequences of real values. To use it one needs to establish a connection between 
real sequences and integer sequences. One way to do it is to use the parking polytope 
introduced in Section 13.4.2. In Section 13.4.3 we provide another proof of Theorem 
13.4.1 via the theory of Gonéarov polynomials. 

The determinant formula in Theorem 13.4.1, while giving a solution for PK,,(u), 
is not easy to compute for a general sequence u. An easy case known is when u 
is an arithmetic progression, i.e., u; = a+(i—1)b for some positive integers a,b. 
In that case PK,(a,a+b,...,a+(n—1)b) =a(a+nb)""!, and the corresponding 
parking functions are called classical parking functions. Many enumerative results 
and combinatorial correspondences described in the previous two sections can be 
extended to classical parking functions, which we will discuss in Section 13.4.4. 

For a sequence u, let A(u) be the difference sequence (u1,u2 — u1,u3 —u2,...). 
Besides classical parking functions, the only PK,,(u) that are computed explicitly are 
the following two cases. 


Theorem 13.4.2 


1. If 
n—m—1 m-1 
A(u) = (a,b,. \b.G,.. 0d), 
then 
ERW) =a y (") (m+1—j)(c—b) [at (nj 
j=0 


1 


-[((m + e= (m-+1 = jb)” + j(d—c)(me—(m+1 io)’*). 
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2. If 
n—m-—1| m1 
A(u) = (a,b, sD dyCjeneg€); 
then 
m(n 
PK, (u) =a y @ [d+ (m— j)c—(m+1 —j)| 
j=0 


-[d-+me—(m+1— jb] "fat (— jo" 7". 3.7) 


Theorem 13.4.2 was proved in [86] by a combinatorial decomposition that partitions 
every integer sequence into two subsequences: a “maximum” parking function and 
a subsequence consisting of terms of higher values. Formula (13.7) with c = 0 was 
also given in [67] by connecting it to the empirical cumulative distributive function 
based on a sample of n independent uniform (0, 1) variables crossing an arbitrary line 
through the unit square. Note that when A(u) = (n— (m—k)+1,1,...,1,0,...,0), 
PK,,(u) is exactly the number of defective parking functions with m drivers, n parking 
spaces, and at most k defects, as described in Section 13.3.5. 

In Section 13.4.2 we introduce a combinatorial representation of a u-parking 
function of length n: it equals the volume of certain polytope in R”, which admits 
a number of interpretations, in terms of empirical distributions, plane partitions, and 
polytopal subdivisions. Then we present in Section 13.4.3 a special sequence of poly- 
nomials, namely, Gonéarov polynomials, that form a natural basis for working with 
u-parking functions. The connection between Gonéarov polynomials and u-parking 
functions was established by extending the decomposition of [86]. Many properties 
of u-parking functions can be derived from the theory of parking polytopes and the 
Gonéarov polynomials. In particular, there are various formulas that allow us to com- 
pute the value of PK,,(u) efficiently. 


13.4.2 A parking polytope 


In [67] Pitman and Stanley introduced an n-dimensional polytope I1,, whose volume 
defines a polynomial that has many combinatorial interpretations. Actually the vol- 
ume polynomial of II, is a variant form of u-parking functions. We present some 
basic properties of the polytope II, and its volume polynomial. 

Let 


j ui 
TI, (x) :={y € R” : y; > 0 and Ys Y? x; for all 1<j<n} 
i=l i=l 
for arbitrary x = (x1,...,%») with x; > 0 for all i. The n-dimensional volume 
Va (x) = Vol(TI,(x)) 


is a homogeneous polynomial of degree n in the variables x,,...,x,, which is called 
the volume polynomial. For example, volume polynomials of small indices are 


Vi (x) = Xi, 
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1 
Vo(x) = x1x2+ 5H 
1 1 1 1 
V3(x) = x1x2%344 sine + 5x1xg + 5xix3 + EX. 


The volume polynomial V,,(x) is closely related to the order statistics of n inde- 
pendent random variables. Let U;,U2,...,U, be m independent random variables 
uniformly distributed in (0,1), and {Ui : i <i <n) be the order statistics of 
U,,U2,...,Uy. Because the random vectors (UW :1<i<n) and (1— Cease 
i <n) have the same uniform distribution with constant density n! on the simplex 


{uER"’:0<u <---<m <1}, 


one obtains that for arbitrary vectors u and r in this simplex 


Pr(Ui) <uj forall 1 < j <n) =nlV,(x1,...,%n) (13.8) 
where (x1,...,Xn) = A(u), or equivalently, uj; = es 1x; and 
Pr(U(j) > rj for all 1 <i <n) =n!Vy(x1,..-,Xn) (13.9) 


where xj = rn42—j —Yn+1—; (with the convention that r,,, = 1). Formulas (13.8) and 
(13.9) allow us to compute V,,(x) using the probability theory. 


Theorem 13.4.3 For a positive integer n, 


7 k kn 


KEK, i= 


where K,, is the set of balanced vectors of length n, that is, 


j n 
K, ={k EN" : ))kj> jforalll <i<n—land Yk =n}. 
i=l i=1 


Proof. Letuj= pa x;. By homogeneity of V,,, it suffices to prove Formula (13.10) 
with uw, < 1. Fix u and consider the probability in (13.8). For 1 <i<n+1 let N; be 
the number of U; that lying in the interval (u;_1,u;], with the conventions that uo = 0 
and uy+1 = 1. 

Since U; are independent and uniformly distributed in (0,1), the random vec- 
tor (Nj: 1 <k <n+1) has the multinomial distribution with parameters n and 
(X1,---;Xn,Xn+1), where Yn tl = 1—u, =1—Y7_, x. In other words, for any sequence 
(k1,.--,kn41) with pee | =n, we have 


n ks n+] y 
Pr[(M,.--,Nn41) = (k1,---,knt1)] = e k ) x =n! TTS. 
: nt j=l “i: 
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Note that N; also equals the number of U/;)’s that lie in (uj_1,u;]. Hence Uy Sui 
if and only if Yi_, N; >i. It follows 


Pr(U(i) <u; for all 1 <i <n) 


= Pr() {Nj >i forall 1 <i<n) 

i=l 
$5 PrN nN Naat) = (yay he, 0) 
keK, 


kj 


= "yee 


keK, i= ik 


Combining the above equation with (13.8) we prove Theorem 13.4.3. a 


From the above proof follow two more probabilistic interpretations of V,, (x). 


Corollary 13.4.4 Let (N;,1<i<n+1) bea random vector with multinomial distri- 
bution with parameters n and (p1,..-,Pn+1), as if N; is the number of times i appears 
in a sequence of n independent trials with probability p; of getting i on each trial for 
1<i<n+l, where yet Di = 1. Then 


Pr (Es >iforall 1 cise] =n!V,(p1,P2,---;Pn); 
j=l 


and 


Pr (Es <iforalll <i< ” =n!V,(Pn+1;Pny-++5P2)- 
j=l 


The relation between V,,(x) and the generalized parking functions is given by the 
following theorem. 


Theorem 13.4.5 Assume that x = (x1,...,X%n) € N". Let uj = ie i.e, x = A(u). 
Then 


PK,(u)= xa, 410+ Xa,41 = n!Vn(X). (13.11) 
ac PL, 
Proof. Given an ordinary parking function a = (a),...,@,), replace each i— 1 by an 


integer in the set {x) +---+x;_1,...,%, +-:-+2x;—1}. The number of ways to do this 
is given in the middle expression in (13.11), and every u-parking function is obtained 
exactly once in this way. This gives the first equality. The second equality follows 
from Theorem 13.4.3 and 13.2.11, as every ordinary parking function a is obtained 
by a balanced vector k € K, which determines that the terms of a contain exactly k; 
(i—1)’s, and a compatible permutation that determines how the terms are arranged. 
Given a balanced vector k € Ky, it is clear that there are ( ee, i) permutations that 


are compatible with k. | 
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In particular, Formula (13.10) gives an explicit way to compute PK,,(u), which is 
much easier than the determinant formula in Theorem 13.4.1. 

An interesting special case of Theorem 13.4.5 is when we take x; = q'—!. In this 
case we have 


AVAL) = VY gtttete = g2)P,(1/q) =q@h(1/9), 
acePKy, 


where P,,(q) is the displacement enumerator of ordinary parking functions, and [,(q) 
is the inversion enumerator of labeled trees. 

The polytope I,,(x) has many interesting properties. For example, it admits a 
subdivision into a collection of n-dimensional chambers, with the volume of each 
chamber corresponding to a term of the volume polynomial. It also relates to plane 
partitions, rooted binary trees, and another polytope called associahedron. More 
combinatorial properties of IT;(x) are discussed in [67]. 


13.4.3 Theory of Gonéarov polynomials 


In this section we describe a polynomial sequence that forms a natural basis for 
working with u-parking functions. The involved polynomials are called Gonéarov 
polynomials, which arose from the Gonéarov interpolation problem in numerical 
analysis. 


[Gonéarov Interpolation] Given two sequences of real or complex 
numbers ao,@1,...,d, and bo,b1,...,bn, find a polynomial p(x) of de- 
gree n such that for each i,0 <i <n, the ith derivative p“) (x) evaluated 
at aj equals ;. 


Gon€éarov polynomials are the basis of solutions to the Gonéarov interpolation 
problem and correspond to the cases that all b; but one are zero. To state their prop- 
erties, we start with a brief introduction of the theory of sequences of polynomial 
biorthogonal to a sequence of linear functionals. The details can be found in [60]. 

Let F be the vector space of all polynomials in the variable x over a field F of 
characteristic zero. Let D: Y — F be the differentiation operator. For a scalar a in 
the field F, let 

e(a): PF, p(x) p(a) 
be the linear functional that evaluates p(x) at a. 
Let ~;(D),s =0,1,2,... be a sequence of linear operators on Y of the form 


Ds (D) = D* y bs-D", 
r=0 


where the coefficients bso are assumed to be non-zero. There exists a unique sequence 
Pn(x),n =0,1,2,... of polynomials such that p,(x) has degree n and 


E(0) s(D) Pn(x) _ n!6sn, 


where 6,, is the Kronecker delta. 
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The polynomial sequence p,,(x) is said to be biorthogonal to the sequence @, (D) 
of operators, or, the sequence €(0)@;(D) of linear functionals. Using Cramer’s rule to 
solve the linear system and Laplace’s expansion to group the results, we can express 
Pn(x) by the the following determinantal formula: 


boo bor bor... bon-1 bon 
O bi by... Dy n-2 Din-1 
n! 0 0 bx ines bo n-3 ba n-2 
Pa boohio***bno |? : : = : : 
O° Oe rok big ae 
Loox x2/2) 2. 1/1)! x*/n! 


Since {pn(x)}”_9 forms a basis of Y, any polynomial can be uniquely expressed 
as a linear combination of p,(x)’s. Explicitly, we have the expansion formula: If 
p(x) is a polynomial of degree n, then 


where d; = €(0);(D) p(x). In particular, 


ji y m\D; n—ipi(x) 


x = 7 
i=0 : 


>) 


which gives a linear recursion for p,(x). Equivalently, one can write the above 
equation in terms of formal power series equations, and obtain the Appell relation 


-yeer Pn n ‘Dont 


where @,(t) =O? oDsrt’. 

A special example of sequences of biorthogonal polynomials is the Gonéarov 
polynomials. Let (ao,a1,a2,...) be a sequence of numbers or variables called nodes. 
The sequence of Gonéarov polynomials 


8n(X340,41,-.-,4n-1), n =0,1,2,... 


is the sequence of polynomials biorthogonal to the operators 


=p y asD" _ )D. 
As indicated by the notation, g,(x;a0,41,...,@n—1) depends only on the nodes 
ao,d1,---,@n—1. In particular, when all the a; equal a, we have 


8n(X34,4,...,4) = (x—a)” 
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and Gonéarov interpolation is just expansion as a power series at x = a. When 
ao,a1,a2,... form an arithmetic progression a,a+b,a+2b,..., we get Abel poly- 
nomials 


8n(X3a,a+b,a+2b,...,a+(n—1)b) = (x—a)(x—a—nb)"!. 


Gonéarov polynomials have many nice algebraic and analytic properties, which 
make them very useful in analysis and combinatorics. Next we list some basic prop- 
erties, of which the first four follow from the theory of biorthogonal polynomials. 


Theorem 13.4.6 The Goncarov polynomials gy(x;a0,41,---,4n—1) have the follow- 
ing properties. 


1. Determinant formula. 


3 n—1 n 

4 4 % % 

1a 3 3 mT! nl 
2 n— n-1 

Ola, oF 4 . 

1 3 (2)! (nT)! 

at a,” 

2 2 
8n(xX340,41,---;An—1) =n! 0 0 1 aw (n—3)! (n—2)! 
0 0 O 1 an-1 

I oe Pe) yn x? 

2 BY (n—1)! n! 


2. Expansion formula. If p(x) is a polynomial of degree n, then 


p(x) = LadP'Pe) 


i! Bi(x3a0,d1,...,ai-1). 
i=0 : 


3. Linear recurrence. Let p(x) =x" in the expansion formula, we have 


n 
x= y (") a; ‘gi(x;a0,41, epig@iet): (13.12) 
i=o \! 
4. Appell relation. 


co 


e = y 8n(x540,41,- ee Ga=7) 
n=0 


teat 


n! 


5. Differential relations. The Gonéarov polynomials can be equivalently defined 
by the differential relations 


Dgn(x3a0,41, on n—1) aan NGn—1(X541,42, oe ,4n—-1), 
with initial conditions 


8n(40340,41,---,4n—1) = O0,n- 
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6. Integral relations. 


xX 
8n(X50,41,-+-,4n—1) = n | 8n—1(t541,42,--.,An—1)dt 
ao 


x ty t—-1 
= nt | dn | dry | dtn. 
ci) Q Q-1 


7. Shift invariance formula. 
8n(x+E3a0+6,a1 iG onentnea +6) = 8n(%340,41,..-,@n—-1)- 
&. Perturbation formula. 


8n(X5a0, 666 Gin—1,Am + Dm Am4i5 ++ jGn1) 


= Bn(X3d0,---Am—1,4m4m41;+++,An—-1) 
n 
_ (27) nln + Bin inst sss) 40:45) 
9. Sheffer relation. 
8n(X +340, - oor »Gn—1) ae y ("") eniosan see »An—1)X'. 
i=0 
In particular, 

n n ? 

8n(X30,---,@n—1) a y (7) eni(O,t dna)! 
i=0 


That is, coefficients of Gonéarov polynomials are constant terms of (shifted) 
Gonéarov polynomials. 


It turns out that u-parking functions provide a combinatorial interpretation of 
Gonéarov polynomials. 


Theorem 13.4.7 Let u = (u1,u2,...) be a sequence of non-decreasing positive inte- 
gers. Then we have 


PK, (u) = PKy(u1,U2,---;Un) = 8n(asx—U1,x—U2,...,X—Un) 
8n(O0; —u1, —U2,..., —Un) 
(—1)"gn(0;u1, U2, tee ,Un). 


Proof. Letx = (x1,%2,..-,X,) be an integer sequence whose terms satisfy 0 < x; <x 
for a positive integer x, and (X(1),*(2)5 wee stn) be the order statistics of x. Let m be 
the maximum index such that 


xq <uj for i=1,2,...,m. (13.13) 
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Then, the subsequence (xj, ,X;,,---;%i,,) from which the sequence (x(1),(2);--+;X(m)) 
was obtained by rearrangement is a u-parking function of length m. Furthermore, m 


is the maximum index satisfying condition (13.13) if and only if 


X(n) 2 X(n-1) eer 2 X(m+1) > Um+1- 


Equivalently, the complementary subsequence (xj, ,Xj.,---,Xj,_» )» obtained by delet- 
ing the subsequence (xj, ,Xi,,-.-,Xi,,) from the original sequence, takes values in the 
interval [un41,x — 1]. Since the maximum index m and the set {i,i2,...,im} are 
uniquely determined by the sequence (x1,x2,...,%,), and any pair of subsequences 
satisfying the conditions in the theorem can be reassembled into a sequence in 
[(0,x — 1]", this decomposition yields a bijection, which leads to the equation 


n 
x= y (*) (x—Ums4i)" "PK m(u1,U2,--.,Um)- (13.14) 


m=o \1 


Comparing the recursion (13.14) with the linear recursion (13.12) for Gonéarov poly- 
nomials, we obtain 


PK, (u1,U2,---,Un) = Sn(4ix— U1, xX —U2,...,X — Un). 


By the shift invariance formula, 


PICS tae Uy ,X —U2,...,X — Un) = 8n(0; —Uj,—U2,..-, Un). 
Since the Gonéarov polynomial g,(x;a0,a1,..-,@n—1) is a homogeneous polynomial 
of total degree n in x,a9,a1,...,4,—1, we have 
8n(0;—u1, —U2,..., —Un) = (—1)" gn (O31, u2,..-,Un)- 


Any reasonable formula for Gonéarov polynomials yields a reasonable formula 
for u-parking functions. For example, when u; = a+ (i—1)b, we obtain the following 
special case. 


Corollary 13.4.8 
PK, (a,a+b,a+2b,...,a+(n—1)b) =a(a+nb)""!. 
The homogeneity of Gonéarov polynomials implies 
Corollary 13.4.9 
PK, (bu, bua, ...,bUn) = b"PK,(u1,u2,..-,Un). 


Using the determinant formula of Theorem 13.4.6, we get a proof of Theo- 
rem 13.4.1, the discrete analogue of Steck’s formula. The two formulas of Theo- 
rem 13.4.2 can be proved by using the perturbation formula of Gonéarov polynomi- 
als. In addition, one could compute the sum enumerator of u-parking functions by 
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applying the properties of Gonéarov polynomials and the decomposition of Theorem 
13.4.7. 
The sum enumerator S,,(q;u) of the set of u-parking functions is the polynomial 


in g defined by 
Sn(q;u) =f y | a oa 
(a ,d2,.-.,4n) 
where the sum ranges over all u-parking functions (a) ,a2,...,d,). The sum enumer- 


ator may be regarded as a “g-analogue” of PK,,(u). 


Theorem 13.4.10 The sum enumerator S;(q;u) satisfies the equation 


“(n 
(ltgt+¢t..tg =P @ (gins gin hE tg Sn (qe). 


m=0 


Proof. Since sum enumerators are multiplicative, the sum enumerator of integer 
sequences (x1,...,%,) with 0 < x; <x is 


(haat page: 


For the same reason, the sum enumerator of sequences that are decomposed into a 
u-parking function of length m and a sequence in [um+1,x — 1]"~"” is 


(qin! f gimtitl fp asaes +q°")""Sin(q,u). 


The recursion now follows. | 


Comparing the recursion in Theorem 13.4.10 with the linear recursion (13.12), 
we obtain 


S(qem) = PR, eg tency lpg het jitter ty), 


Using Theorem 13.4.7 and the shift invariance formula, we can express sum enu- 
merators in terms of Gonéarov polynomials, as 


I ee, Ge i 
Sn(q;u) = : ; pany . 
n(q u) o (i i—¢'1—¢ i= 


By homogeneity of Gonéarov polynomials, 


(1 —)"Sn(qsu) = 8n(1g",q'",...,q""). 
Hence, sum enumerators satisfy the simpler linear recursion 


n 


it —m m 
l= y ("ate )(1—q) Sin(q;u). (13.15) 


m=0 
They also satisfy the following Appell relation 
co n 


exp(t) = ¥ (1—4)"Sn(q;u) exp( q's) — 


= 
n=0 ue 
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In the case of ordinary parking functions with u; = i, we have 


qd GN nt ta daawsglt) Spligig ig’ ys 


and the sum enumerator is related to the displacement enumerator of ordinary park- 
ing functions by the equation 


Sn(qi1,2,-.-.n) =g2)P,(1/q). 


Hence (13.15) leads to a linear recurrence for P,,(q): 


n 
ies gla)". (g—1)™P,,(q). 
m=0 

More properties and computation of parking functions via Gonéarov polynomi- 
als are given in [59, 60]. For example, [60] gives the generating functions for fac- 
torial moments of sums of u-parking functions, while the explicit formulas for the 
first and second factorial moments of sums of u-parking functions are given in [59]. 
Khare, Lorentz and Yan [49] studied multivariate Gonéarov polynomials, and ex- 
tended many algebraic and analytic properties of Gonéarov polynomials to the mul- 
tivariate case. They also established a connection between multivariate Goncarov 
polynomials and order statistics of integer sequences, which leads to a higher dimen- 
sional generalization of parking functions. 

If one changes the differential operator D to the backward different operator A 
that maps a polynomial p(x) to Ap(x) = p(x) — p(x— 1), then one gets the difference 
Gon€arov polynomials, whose combinatorial counterpart is the set of lattice paths 
with a given right boundary. Theory of difference Gonéarov polynomials is presented 
in [57]. 


13.4.4 Classical parking functions 


A special class of u-parking functions is the classical parking functions, for which 
the entries of the vector u form an arithmetic progression. In particular, if u; = 
a+(i—1)b for some positive integers a and b, we will call the u-parking func- 
tions (a,b)-parking functions. Classical parking functions have a rich theory. Many 
combinatorial representations of ordinary parking functions can be generalized to 
this case, much in the oar way as theory of Catalan structures (structures counted 


by the Catalan numbers => ) ) generalizes to that of the Fuss-Catalan structures 


(structures counted by the Fuss-Catalan numbers, or k-Catalan numbers = Cera) 
In this section we give a brief summary of such generalizations. 


Basic enumeration. It is well-known that the number of (a,)-parking functions is 
a(a+nb)""}, 

Let E;(n;a,b) be the expected value of the kth factorial moment of the sum of 
the terms in a uniform random (a,b)-parking function, that is, 


1 


Ex(n;a,b) = a(a-tnby=t 


y (xy tx2 +... +4n)k, 


(x1 X25...) 
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where (n), is the falling factorial n(n — 1)---(n —k+1) and the sum ranges over 
all (a,b)-parking functions (x),x2,...,%,). Explicit formulas for E)(n;a,b) and 
E>(n;a,b) are computed by Kung and Yan in [58, 59]. 


Theorem 13.4.11 The expected sums of terms in a random (a,b)-parking function 
of length n is 


b+1) 1 i\b/ 
E(nia,b) = "ERED Fy (") J 


i) (at+nb)i- 


Theorem 13.4.12 The second factorial moment of the sum of terms of a random 
(a,b)-parking function of length n is 


In(n—1)(a+nb+1)? + 4n(a+nb+1)(a+nb—1) 
n(atnb+1) 4 (n jlbi n(n jlbi ba sae sl 
2 Y arth i) Gee 6 ase 


j=l Jal 


Kung and Yan also presented a general form for the higher moments of the ex- 
pected sum of (a,b)-parking functions, see Theorem 1.1 of [58]. 


Rooted b-forests. The correspondence between ordinary parking functions and la- 
beled trees can be extended to (a,b)-parking functions, where the labeled trees are 
replaces by forests of labeled trees with edge colors. Precisely, define a rooted b- 
forest on |r] to be a rooted forest on the vertex set [n] whose edges are colored with 
the colors 0,1,...,5— 1. There is no further restriction on the possible coloring of 
the edges. Let ¥,,(a,b) be the set of all sequences (1, 72,...,T) of length a such 
that 


(1) each T; is a rooted b -forest, 
(2) Tj and T; are disjoint if i ~ j, and 
(3) the union of the vertex sets of T),...,Tz is [n]. 


Denote by ¥.%,,((a,b)) the set of (a,b)-parking functions of length n. Then 
there is a one-to-one correspondence between Y.%;,((a,b)) and ¥,,(a,b). In fact, 
both of them can be mapped bijectively to the set G, (a,b) that consists of pairs (7,0) 
such that 


(1) F= (ro.r1,---sTas(n—1p-1) € NeF@-) is (a,b)-balanced, that is, 


rotritet+ rasib -1 >1+1 fori=O0,1,...,n—-2, 


TOTTI Tt Tat (n—1)b-1 = M- 
(2) o € G, is compatible with 7, that is, the terms in the inverse o! of o is 
increasing on every interval of the indices {1 = ya rj,2 +ye, Ti ons eo ri} 
(if rp41 #0). 
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These results were proved in [87] by using the breadth-first search on rooted b- 
forests, which generalize Theorems 13.2.11 and 13.2.12. Eu, Fu and Lai [27, 28] 
applied the construction of [87] to enumerate (a,b)-parking functions by their lead- 
ing terms, or with certain symmetric restrictions and periodic restrictions. 


Inversions of b-forests and multicolored graphs. The notion of inversions can be 
extended to the set F,(a,b) as follows: Let F = (M%,7Tb,...,T) be a sequence of 
rooted b-forests on |[n]. Denote the color of an edge e by k(e). Define the (a,b)- 
inversion inv'“”) (F) by letting 


a 


inv) (F) =inv(F)+ VP (i-D|T|+ YY Ke), 


i=l x€[n] e€ K(x) 


where inv(F’) is the number of inversions of T; UT)---U 7, as an ordinary rooted 


forest, K(x) is the set of edges lying between the vertex x and the root of the unique 


tree to which x belongs. Define the (a, b)-inversion enumerator 10) (q) by 


a, inv(@2) 
AC) ee ee aaa a 


n 
FEF, (a,b) 


For a= 1, 1{*”) (q) = 1h?) (q) is the b-inversion enumerator studied in [81, 85], and 


fd) (q) = 1,(q) is the ordinary inversion enumerator of labeled trees. 
For an (a,b)-parking function a = (a1,...,dy), the (a,b)-displacement D‘“) (a) 


is defined as 
D(@) (a)=b (5) +an— LX di, 
and the (a,b)-displacement enumerator is 
ib (a,b) 
Pg= Yo Pre. 
ac PH y((a,b)) 


Then we have 


Theorem 13.4.13 ; ‘ 

TOG =P (a): 
Both of the above polynomials are related to another polynomial ce) (q) that counts 
the number of edges in connected multigraphs. To wit, define a multicolored (a,b)- 
graph on [n] to be a graph G with the vertex set [n] such that 


(1) The edges of G are colored with colors 0,1,...,b—1, 


(2) There are no loops or multiple edges of the same color in G. But G may have 
edges with the same endpoints but different colors, and 


(3) every vertex r is assigned with a subset f(r) of [a] = {1,2,...,a}. We say that 
risa root of Gif f(r) 40. 
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(4) For any subgraph H of G, define R(H) = ¥,cy |f(r)| to be the number of roots 
in H, counting multiplicity. Every connected component G’ of G has at least 
one root, i.e., R(G’) > 0. 


Denote by e(G) the number of edges of G, and by r(G) = ¥;,|f(r)| the number of 


roots of G. Let (ab) 
ce (q) = yee 
G 


where G ranges over all multicolored (a,b)-graphs on [n]. Set ce) (q) = 1 for all 


a,b EN. Then 
Theorem 13.4.14 


i?) (1 +4) = PM (144) =c(g). 


Theorem 13.4.14 is proved in [87] by applying the DFS and BFS to multicolored 
graphs. The case that a = | was proved earlier in [81, 85] by recurrence relations. In 
[75] Shin and Zeng constructed a bijective proof for the case that a is a multiple of 
b, which gives a refinement of Theorem 13.4.13 in that case. 


b-divisible noncrossing partitions. Theorems 13.3.2 and 13.3.4 have been general- 
ized by Stanley [79] to the lattice of b-divisible noncrossing partitions. Fix an posi- 
tive integer b. A b-divisible noncrossing partition is a noncrossing partition 7 for 
which every block size is divisible by b. Thus z is a noncrossing partition for a set 
[bn] for some n > 0. Let nc? be the poset of all b-divisible noncrossing partitions 


of [bn]. The combinatorial properties of the poset M' cP were first studied by Edel- 


man [26]. In particular, if a pair (7, 7’) is an edge of NCW), then (mz, 7’) is an edge of 
NCpn. Hence the edge labeling A and A, of NC, defined in Section 13.3.2 restrict to 
edge-labelings of NM oe 

Theorem 13.3.2 is generalized as follows. 


Theorem 13.4.15 The label A,(m) of the maximal chains of NC”), consists of the 
(b,b)-parking functions of length n, each occurring once. 


The posets nc) do not have a 0 when b > 1. For these posets one regards 
the minimal elements as having rank 0, and defines 4, (») (S) and Bw) (S) for 
n+1 n+l 

SC [n—1]o. Theorem 13.3.4 is generalized as follows. 


Theorem 13.4.16 Let S C [n— 1]. 


1. The number of |b,b|-parking functions a of length n with Des(a) = S is equal 
to 


Byow (ln —8) + Byew (ln— Ho — 5). 


2. The number of |b,b]-parking functions a of length n satisfying S C Des(a) is 
equal to 
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Extended Shi arrangements. For b > | the extended Shi arrangement we is the 
collection of hyperplanes 


xj—xj =—b+1,—-D4+2,...,b, forl <i<j<n. 


The labeling of the regions of the Shi arrangement defined in Section 13.3.3 can be 
extended to this case, as described by Stanley [81]. 
Define the base region Ro of 4? by 


Ro: x1 > X2 > ++: > Xp, > xy — 1. 


First label the region Ro by (0,0,...,0) € N”. Suppose now that R has been labeled, 
and that R’ is an unlabeled region that is separated from R by a unique hyperplane 
H : x; —xj; =m, where i < j. Then define 


A(R’) = A(R)+e;, If H is given by x; -—xj =m withi < jandm>0 
~ | A(R)+e;, if H is given by x;-xj =m withi < jandm<0. 


Then 


Theorem 13.4.17 The labels A(R) of the extended Shi arrangement SF) are just 
the (1,b)-parking functions of length n, each occurring exactly once. In addition, the 
distance enumerator of the extended Shi arrangement is equal to the sum enumerator 
of (1,b)-parking functions. 


13.5 Parking functions associated with graphs 
13.5.1. G-parking functions 


In 2004 Postnikov and Shapiro [68] proposed a new generalization of parking func- 
tions, namely, the G-parking functions associated with a general connected digraph 
G. Let G be a directed graph with the vertex set [n]o, where multiple edges and loops 
are allowed. We will view the vertex O as the root. As usual in graph theory, a di- 
rected edge is represented by a pair (i,j) of vertices, where i is the tail of the edge 
and j is the head of the edge. For a vertex i, the indegree indeg(i) is the number of 
edges with tail i, and the outdegree outdeg(i) is the number of edges with head i. 
In addition, for any subset U C [n] and vertex i € U, we define outdeg,(i) to be the 
cardinality of the set {(i, 7) € E(G) | j €U}, where E(G) is the set of edges of G. 


Definition 13.5.1 A G-parking function is a function f from {n] to the set of non- 
negative integers N satisfying the following condition: For each subset U C {n|\ of 
vertices of G, there exists a vertex i € U such that f (i) < outdeg;, (i). 


For the complete graph K,,; on |[n]o, such defined functions are exactly the ordinary 
parking functions if we view K,+, as the digraph with one directed edge (i, j) for 
each i # j, and record the function f as a sequence (f(1), f(2),...,f(n)). 
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In a digraph G an oriented spanning tree T of G is a subgraph T in which 
(1) the root 0 has outdegree 0, (2) all other vertices have outdegree 1, and (3) there 
exists a unique directed path from any vertex i to the root 0. The number of oriented 
spanning trees of G, denoted «(G), is sometimes called the complexity of G. The 
value of k(G) can be computed by the Matrix-Tree Theorem; see [78, Chap 5.6]. 

The first important result of G-parking functions is the following theorem that 
extends the relation between ordinary parking functions and labeled trees. 


Theorem 13.5.2 The number of G-parking functions equals the number of oriented 
spanning trees of the digraph G. 


The motivation for Postnikov and Shapiro to work with G-parking functions is 
to study a general class of algebras formed by taking the quotient of the polyno- 
mial ring modulo monotone monomial ideals and their deformations. As a special 
example, for any digraph G they define two algebras .% and Ag and describe their 
monomial bases. The basis elements correspond to G-parking functions. Then they 
proved Theorem 13.5.2 by applying general formulas for the Hilbert series and di- 
mensions of the algebras given by a monotone monomial ideal. 

Shortly after [68] appeared, Chebikin and Pylyavskyy [21] constructed a family 
of bijections between the set of G-parking functions and the set of oriented spanning 
trees of the digraph G. Their bijections were built on “different total orders” on the 
vertices of subtrees of G, similar to the “selection procedure” of Frangon’s [31] in 
constructing bijections between ordinary parking functions and rooted labeled trees. 
We will describe these bijections in a slightly more general setting in Section 13.5.3. 


13.5.2 Abelian sandpile model 


There is an interesting relation between G-parking functions and the abelian sand- 
pile model, one of the archetype models of self-organized criticality in physical sys- 
tems. The sandpile model was originated from an automaton model by Bak, Tang, 
and Wiesenfeld [7] for evolutions of dynamical systems on Z?, and was first formu- 
lated by Dhar [25] on any finite connected graphs. The model was also considered 
by combinatorialists as a game on a graph called the chip firing game or the dollar 
game, e.g. [12, 14, 15]. 

To show its connection to Tutte polynomials, we describe a simple version of the 
sandpile model for undirected graphs with possible multiple edges but no loops. A 
review of the more general theory with a class of toppling matrices, as well as the 
history and references on the subject can be found in the appendix of [68]. 

Let G be a connected graph on the vertex set [n]o, where the vertex 0 is the root. A 
configuration (sandpile) is a collection of indistinguishable chips distributed among 
the vertices of G. More precisely, it is a function u from [nm] to N indicating how 
many chips are at each vertex i 4 0. A non-root vertex i is called unstable if it has at 
least as many chips as its degree, and an unstable vertex can topple by sending chips 
to adjacent vertices, one along each incident edge. In formula, toppling at vertex 7 
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changes a configuration u to u’ by 


yi n={ u(i)—deg(i) if j=i, 


u(j)+e(i,j) otherwise, 


for j 40, where e(i, j) is the number of edges between i and j. 

In a configuration toppling one vertex may cause neighboring vertices to become 
unstable. A configuration is said to be stable if all the non-root vertices are stable. It is 
easy to see that for a connected graph G any initial configuration can be transformed 
into a stable configuration by a sequence of topplings at non-root vertices. Moreover, 
the final stable configuration does not depend on the order of the topplings, leading 
to the name “abelian sandpile model.” 

The root 0 plays a special role, for which we set that u(0) = —Y;z9u(i). For 
a stable configuration u, we can let the root topple by increasing the chips at any 
neighbor i of 0 by e(0,i), where e(0,i) is the number of edges connecting 0 to i, and 
deceasing the “chips” at the root by deg(0). By allowing the root to topple for stable 
configurations, the sandpile model can continue indefinitely, and produce an infinite 
number of stable configurations. A configuration is recurrent in an evolving system 
if it could be observed after a long period of the evolution of the system. For abelian 
sandpile model, the system is considered to evolve by adding some chips at a random 
vertex and then applying the toppling rules. This leads to the following definition. 


Definition 13.5.3 A configuration u is recurrent if it is stable and there exists a 
positive configuration v £ 0 such that u can be obtained by a sequence of topplings 
from u+v. 


Recurrent configurations are also called critical configurations in [12]. There are 
many characterizations of recurrent configurations, see, for example, [25, 12, 24, 22]. 
The following explicit characterization is due to Dhar. Let us say that a configuration 
u is allowed if for any non-empty subset J C [n] there exists j € J such that 


u(j)> YY efi, J). 
iel\{j} 


Theorem 13.5.4 A configuration is recurrent if and only if it is stable and allowed. 


Comparing Theorem 13.5.4 and the definition of G-parking functions, we see that 
recurrent configurations and G-parking functions are complementary to each other, 
as pointed out in [68]. The special case for complete graph K,1 was proved earlier 
by Cori and Rossin [24]. 


Theorem 13.5.5 Let G be a connected graph on |n\o. A configuration u : [n| > N is 
recurrent if and only only if the function uY given by uX (i) = deg(i) — 1 —u(i) is a 
G-parking function. 


Dhar described a simple recursive procedure, the burning algorithm, to deter- 
mine if a given stable configuration is allowed. Translated to G-parking functions, 
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it leads to a linear algorithm that tests whether a given function u : [n] > N is a G- 
parking function: First we mark the root 0. At each iteration of the algorithm, we 
mark all vertices i that have more marked neighbors than the value u(i). Then u is a 
G-parking function if and only if all the vertices are marked in the end. 

The following example shows how the algorithm works. In Figure 13.8 there are 
two vertex-functions on the complete graph Ks, where each vertex i # 0 is labeled 
by i/f(i), ie., the first label is the index of the vertex, and the second label is the 
function value. The first one shows a G-parking function, as one can mark all the 
vertices in the order 0,1,4,2,3, while the second is not: Once one marks vertices 
0,1,4, no more vertices can be marked. 


1/0 4/0 1/0 4/0 


2/2 3/3 2/3 3/3 


Figure 13.8 
Dhar’s burning algorithm. 


The set of recurrent configurations of a sandpile model on G forms an abelian 
group whose order is equal to the number of spanning trees of the graph. In particular, 
let the level of a recurrent configuration u be given by 


level(u) = + u; + deg(0) — |E(G)|. 


i€(n| 


Written in terms of G-parking function, the statistic level(u) induces a weight of uY 
as 


wt(u’) =|E(G)|—n— Pw. 


Let P(G,y) be the level enumerator of the recurrent configurations of the sand- 
pile model of G, or equivalently, the weight enumerator of G-parking functions, in 


formula, ; 
P(G,y) = ye = yy ) 
u uv 


where u ranges over all recurrent configurations. The polynomial P(G, y) is the natu- 
ral generalization of the displacement enumerator of the ordinary parking functions, 
and is a specialization of the Tutte polynomial Tg(x,y), which we recall next. 
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Suppose we are given a connected graph G and a total ordering of its edges. 
Consider a spanning tree T of G. An edge e € G—T is externally active if it is the 
smallest edge in the unique cycle contained in T Ue. An edge e € T is internally 
active if it is the smallest edges in the unique cocycle contained in (G—T) U {e} (a 
cocycle is a minimal edge cut). We let ea(T) be the number of externally active edges 
of T, and ia(T) the number of internally active edges in T. Tutte [83] then defined 


his polynomial as 
To(x,y) = Y, xiaMyeal?) 
TCG 


where the sum is over all spanning trees T of G. Tutte showed that Tg (x,y) is well- 
defined, i.e., independent of the total ordering of the edges of G. 
The next theorem reveals the relation between P(G,y) and the Tutte polynomial 


Tg (x,y). 


Theorem 13.5.6 For a connected graph G, 
P(G,y) = Te(1,y) 


where Tg(x,y) is the Tutte polynomial of G. In particular, there are as many G- 
parking functions of weight i as spanning trees of G with external activity i. 


Theorem 13.5.6 was conjectured by Biggs and proved by Merino Lopez [64] by 
using a recursive characterization of Tutte polynomials. Cori and Le Borgne [23] 
constructed a bijection between recurrent configurations and spanning trees of G, 
which carries external activities of the spanning trees to the levels of the recurrent 
configurations of the sandpile model. 


13.5.3 Multiparking functions, graph searching, and the Tutte 
polynomial 


To get a complete expression of the bivariate Tutte polynomial, Kostic and Yan [54] 
introduced the notion of G-multiparking functions, which are in one-to-one corre- 
spondence with spanning forests of the graph G. They described a family of bijec- 
tions between the spanning forests of a graph G and the G-multiparking functions, 
each of which corresponds to a graph searching algorithm. In particular, the bijec- 
tion induced by the breadth-first search leads to a new characterization of external 
activity, and hence a representation of the bivariate Tutte polynomial by statistics of 
multiparking functions. This generalizes Theorems 13.2.6 and 13.5.6, as well as the 
correspondence between ordinary parking functions and labeled trees. 

For simplicity and clarity, in the following we assume that G is a simple con- 
nected graph on |n]. The treatment of general directed graphs can be found in [54]. 


Definition 13.5.7 Let G be a simple graph with V(G) = [n]. A G-multiparking 
function is a function f : V(G) = [n] > NU {9}, such that for every U C V(G) 
either (A) i is the vertex of smallest index in U and f (i) =, or (B) there exists a 
vertex i € U such that 0 < f (vj) < outdeg,, (i). 
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The vertices that satisfy f(i) =o in (A) will be called roots of f and those that satisfy 
(B) (in U) are said to be well-behaved in U. The G-multiparking functions with only 
one root (which is necessarily vertex 1) are equivalent to G-parking functions defined 
by Postnikov and Shapiro. 

An easy way to check whether a function is a G-multiparking function is to mod- 
ify Dhar’s burning algorithm to allow the root vertices. 


Proposition 13.5.8 A vertex function is a G-multiparking function if and only if there 
exists an ordering 1(1),7(2),...,%(n) of the vertices of the graph G such that for 
every j, 1(j) satisfies either condition (A) or condition (B) in U;:={(j),...,a(n)}. 


Let “Pg be the set of G-multiparking functions and 4G the set of spanning 
forests of G. 


Theorem 13.5.9 The set @ Ag is in one-to-one correspondence with the set FG. 


Again there are many ways to construct bijections from 4 Ag to Fg. Extending 
the idea of Section 13.2.3, we introduce a family of such bijections, each determined 
by a choice function and corresponding to a searching algorithm on trees/forests. 

For a graph G, a sub-forest F of G is a subgraph of G without cycles. (Here 
we don’t assume any root in F). A leaf of F is a vertex v € V(F) of degree 1| in F. 
Denote the set of leaves of F by Leaf(F). Let [J be the set of all ordered pairs (F,W) 
such that F is a sub-forest of G, and 0 4 W C Leaf(F). As in Section 13.2.3 define a 
choice function ¥ as a function from J] to V(G) such that y(F,W) € W. 

It is easier to describe the bijection from 4g to. W Ag, hence we present it first 
in the following algorithm. Fix a choice function y. Let G be a graph on [n] with a 
spanning forest F. Let 7T,...,7; be the trees of F with respective minimal vertices 
rp=l<n<---<rp. 


Algorithm 13.5.10 


e Step 1. Determine a processing order 2. Define a permutation 1% = 
(x(1),2(2),...,2(n)) = (viv2...Vn) on the vertices of G as follows. First, 


v) = 1. Assuming v,V2,...,vj are determined, 
-— If there is no edge of F connecting vertices in V; = {v1,v2,...,vi} to 
vertices outside Vj, let vis, be the vertex of smallest index not already 
in Vj; 


- Otherwise, let W = {v € V;: v is adjacent to some vertices in V;}, and 
F' be the forest obtained by restricting F to V; JW. Let vj41 = y(F',W). 


e Step 2. Define a G-multiparking function f = fr. Set f(r1) = f(r2) =--- = 
f(r) =. For any other vertex v, let r, be the minimal vertex in the tree 
containing v, and v,v? ,u,,...,Ut,1y be the unique path from v to r,. (That, is, 
vP = pre(v). ) Set f(v) to be the cardinality of the set 


{vjl{vvj} © E(G), wh (vs) <a 1(v?)}. 
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The inverse bijection is quite complicated, which is achieved by the following 
algorithm. Given a G-multiparking function f € WAG, Algorithm 13.5.11 finds a 
spanning forest F € ¥g. Explicitly, we define quadruples (val;, P;, Q;, F;) recursively 
for i =0,1,...,n, where val; : V(G) — Z is the value function, P, is the set of pro- 
cessed vertices, Q; is the set of vertices to be processed, and F; is a sub-forest of G 
with V(F;) = P;UQ;, Q; C Leaf(F;) or Q; consists of an isolated vertex of F;. 


Algorithm 13.5.11 
e Step 1: initial condition. Let valo = f, Po be empty, and Fy = Qo = {1}. 


e Step 2: choose a new vertex v. At time i > 1, let v = y(Fi-1, Qj-1), where y 
is the choice function. 


e Step 3: process vertex v. For every vertex w adjacent to v and w ¢ P,-, set 
val;(w) = val;_1(w) — 1. For any other vertex u, set val;(u) = val;_,(u). Let 
N = {w|val;(w) = —1, val;_1(w) 4 —1}. Update P,, Q; and F; by letting 


1 P= Py fy}, 

2. Q;=Q;-1UN \ {v} if Q;-1UN \ {v} 4 O; otherwise Q; = {u} where u 
is the vertex of the lowest-index in [n] — P,. 

3. Let F; be a graph on P;U Q; whose edges are obtained from those of 
F;_, by joining edges {w,v} for eachw EN. 


We say that the vertex v is processed at time i. 


Iterate steps 2-3 until i =n. We must have P, = {n| and Q, = 0. Define ® = By Gg: 
MPG #¢ by letting B(f) = Fh. 


Note that the forest F = B(f) is built tree by tree by Algorithm 13.5.11 That is, 
if 7; and 7; are tree components of F' with roots r;, rj; and r; < rj, then any vertex 
of tree J; is processed before any vertex of T;. The bijection ® maps G-multiparking 
functions with k roots to spanning forests with k components. 

We explain Algorithms 13.5.10 and 13.5.11 with an example where the choice 
function is given by the breadth-first order with a queue structure, i.e., the order <p sg 
described in Example 13.2.17, with additional rules comparing vertices in different 
trees. Explicitly, we view the minimal vertex of each tree component as the root of 
the tree. The order <p,, is defined as follows. 


1. Any vertex of tree 7; is less than any vertex of tree 7; if the root of 7; is less 
than the root of 7; 


2. For vertices i, j in the same tree component, let i <pyg j if 


(a) level(i) < level(j), or 
(b) level(i) = level(j) and pre(i) <pq pre(j), or 
(c) pre(i) = pre(j) andi < j. 
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Example 13.5.12 A graph Gand a multiparking function f are given in Figure 13.9, 


in which each vertex is labeled by i/ f (i), i.e., the first label is the index of the vertex, 
and the second label is the function value. 


I/co 3/0 


2/0 4/2 


6/1 5/2 


710 


Figure 13.9 
A graph and a multiparking function. 


The spanning forest created by Algorithm 13.5.11 is indicated by the thick lines 
in Figure 13.10, while other edges are in dotted lines. In Figure 13.10, each vertex is 
labeled by i/val,,(i). The (P;,Q;,F;) for this instance are as follows, where each Q; is 
an ordered set, and the first element in Q; is the next one to be processed. Since P,; 
and F; are increasing sets, we just indicate when a new element is added to them. 


t 0 1 2 3 4 ) 6 |7 
Q: (1) (2,3) (3,6) | (6,4) (4,5,7) (5,7) | 7) | 9 
new / in P; 1 2 3 6 4 yeallars 
new edges {1,2}, {1,3} | {2,6} | {3,4} | {6,5}, {6,7} 
in F, 


Conversely, starting with a spanning tree, we can easily recover the G- 
multiparking function using Algorithm 13.5.10. For the spanning tree in Fig- 
ure 13.10, the processing order is 1,2,3,6,4,5,7. We have that f(1) =o since it 
is the minimal element. Let v be the vertex 5. Then v? = 6 and v is connected to two 
other vertices, namely, vertex 2 and 3, which are less than 6 in the processing order. 
Hence f(5) = 2. Similarly we can compute f (i) for other vertices, and get 


FUL) =e, F(2) = F(3) = F(7) =0, F(4) = F(S) = 2, F(6) =1. 


To get a connection to the Tutte polynomial of G, we use another expression 
of Tg(x,y). Let H be a (spanning) subgraph of G. Denote by c(H) the number of 
components of H. Define two invariants associated with H as 


o(H)=c(H)—-1,  o*(H) =|E(H)|—|V(G)| +c(#). (13.16) 
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1/c0 3/-2 


Figure 13.10 
A spanning forest found by Algorithm 13.5.10. 


The following alternative description of Tutte polynomials is well-known, for exam- 
ple, see [11]. 


Theorem 13.5.13 


tg(L+x,1ty)= YF xeDyo, (13.17) 


where the sum is over all spanning subgraphs H of G. 


For a subgraph H of G, we apply the breadth-first search by using the imple- 
mentation with a queue structure, as described in Section 13.2.3, to find a spanning 
forest. Since H may not be connected, we modify the algorithm by adding the least 
unvisited vertex to the queue whenever the queue is empty but not all the vertices are 
processed. By the same argument as before, we partition all the subgraphs of G into 
intervals, one for each spanning forest F’. Each interval turns out to be a Boolean al- 
gebra consisting of all the ways to add some extra edges to its representing spanning 
forest. Expressing the Tutte polynomial in terms of sums over such intervals leads 
to an equation between the Tutte polynomial and a bivariate generating function of 
G-multiparking functions. 

Given a G-multiparking function f, let 7 = 7|7%...7, be an ordering of the 
vertex set [n] determined by applying the burning algorithm in a greedy way: Let 
m, = 1. After determining 7,...,7;_1, if Vi; =V(G) —{m,...,7);-1} has a well- 
behaved vertex, that is, a vertex v € V; such that 0 < f(v) < outdegy, (v), let 2; be one 
of them; otherwise, let 7; be the minimal vertex of V;. In the latter case 7; has to be a 
root of f. 

Define the total root record Rec(f) of f as the cardinality of the set 


{{v,w} € E(G) | vis aroot of f and w appears in front of v in the order  }. 


It can be proved that the value Rec(f) is independent of the order z and hence well- 
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defined. Let the weight of a G-multiparking function f be 


wt(f) = |E(G)|+r(f) —n—Ree(f) — (LF). 
v#0 


where r(f) is the number of roots of f. Then we have 


Theorem 13.5.14 For any connected graph G, 


To(ltxy)= YP x A-ly(f) 
fem FPG) 


where M P(G) is the set of G-multiparking functions on G. 


Note that if r(f) = 1, the function f can only have one root at vertex 1, therefore 
Rec(f) = 0 and wt(f) = |E(G)| —n+1— (41 f(v)). Hence Theorem 13.5.6 is a 
special case of Theorem 13.5.14 by taking x = 0. 


Corollary 13.5.15 For connected graph G, the number of G-multiparking functions 
is tg(2, 1). Among them, those with an odd number of roots is counted by 5 (te (2, 1)+ 
tg(0,1)), and those with an even number of roots is counted by 5 (ta(2, 1)—tg(0,1)). 


Proof. The first sentence is obtained by taking x = y = 1 in Theorem 13.5.14. 
For the second sentence, one notices that tg(0,1) = EP (-Vry )-1 is the difference 
between the number of G-multiparking functions with an odd number of roots, and 
those with an even number of roots. i 


Theorem 13.5.14 also leads to a new expression of fx, (x,y), the Tutte poly- 
nomial of complete graphs with n+ 1 vertices, in terms of ordinary parking func- 
tions. The exponent of the variable x enumerates ordinary parking functions by 
the number of critical left-to-right maxima. Given an ordinary parking function 
b = (bj,...,bn), we say that a term b; = / is critical if in b there are exactly j terms 
less than j, and exactly n — 1 — j terms larger than j. For example, in b = (3,0,0,2), 
the terms b; = 3 and b4 = 2 are critical. Among them, only b; = 3 is also a left-to- 
right maximum. 

Let @(a) be the number of critical left-to-right maxima in an ordinary parking 
function a. We have 


Theorem 13.5.16 
Ky (GY) = x(a) y(2)-Zibi, 
acPH 
where PX , is the set of ordinary parking functions of length n. 


An interesting question is how to express the bivariate Tutte polynomials in terms 
of G-parking functions only, avoiding introducing roots or using spanning forests. An 
answer was provided by Chang, Ma and Yeh [19], who proposed a notion of critical 
bridge for G-parking functions, and proved that 


To(x,y) = xy"), 
7 
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where cb(f) is the number of critical bridges of f, and the sum ranges over all G- 
parking functions. 


13.6 Final remarks 


We close this survey by briefly mentioning a fast growing area, the combinatorial 
study of Macdonald polynomials, in which parking functions play an important role. 
Macdonald polynomials [63] is a family of multivariable orthogonal polynomials 
with applications to a wide variety of subjects, including algebraic combinatorics, 
harmonic analysis, Hilbert schemes, and representation theory. The combinatorics 
theory behind Macdonald polynomials was pioneered by Garsia, Haiman, and their 
school, and was developed dramatically in the last 20 years, with exciting new prob- 
lems and progressions. Parking functions and their encoding as labeled Dyck paths 
provide powerful tools to study the intrinsic combinatorial structures. Conversely, the 
theory of Macdonald polynomials leads to many new generalizations and extensions 
that greatly enrich the theory of parking functions. 

A comprehensive coverage on the combinatorics of Macdonald polynomials and 
the space of diagonal harmonics is given in the monograph [43] by Haglund. The enu- 
merative results associated to statistics on parking functions and labeled lattice paths, 
as well as the two-parameter versions of related objects such as Catalan numbers and 
Schroder paths, are included in the chapter Catalan Paths and q,t-enumeration of 
this book. Here we give a gentle introduction of the role played by parking functions 
and the connection to previous sections. 

Parking functions appear in the spaces of diagonal harmonics and diagonal coin- 
variants, which concern the action of the symmetric group G,, on two sets (x1,...,n) 
and (y1,..-,¥,) of n variables. For a polynomial in R, = C[x1,...,%n,¥1,---;Yn], let 
G,, act diagonally, i.e., 


W° Xi = Xyw(i) W- Yi = Yw(i)- 


It is known that 


{¥ xtyh h+k>0} (13.18) 
i=1 


generate R©”, the ring of invariants under the diagonal action. The quotient ring 
DR,, of diagonal coinvariants is defined by DR, = R,,/I, where I is the ideal of R, 
generated by elements of (13.18) with zero constant term. By analogy the space of 
diagonal harmonics DH, is defined by 


n at ak 
DH, =4fERn : Ysa H0,VAt+k>0 : 


i=l i 7] 


The spaces DR, and DH, are finite dimensional isomorphic vector spaces. A major 
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theorem of DH, is the so-called (n + 1)"~! conjecture, which was proposed by Garsia 
and Haiman [34] and later proved by Haiman [46]. 


Theorem 13.6.1 The dimension of DH, is (n+1)""!. 


Let Vj; xn be the set of polynomials f € R, such that f is homogeneous of degree h in 
the x-variables and homogeneous of degree k in the y-variables. By convention, the 
zero polynomial belongs to every Vj; 4. We have the decomposition 


Rn = BDOViin, 


h>0k>0 


which turns R,, into a doubly graded vector space. One can write 


DEn = BB(PA OV ik): 


h>0k>0 


The Hilbert series of diagonal harmonics is the bivariate polynomial 


An(q,t) = y y dim(DH, Waren qt. 
h>0k>0 


Similarly, one can turn DR, into a doubly graded vector space. Ignoring the ring 
structure of DR, the spaces DR, and DH, are isomorphic as doubly graded G,,- 
modules. 

Haiman [46] proved that the Hilbert series H,,(¢,t) has the following properties. 


(). A,(1,1) =(#+1)""1. 
(2) gD? H,(1/q,q) =(It+qt-- tq"). 
(3) An(q,t) = An(t,q). 


By the first property, H,,(q,t) is a sum of (n+ 1)"~! monomials q't*. This suggests 
that there could be a combinatorial interpretation of H,,(q,t) as a joint enumerator of 
two statistics over parking functions. More precisely, one seeks two functions stat, 
and stat2 from Y.%, to N such that 


ae (q,t) = y fone pene), 
AEP KH py 


The first conjectured combinatorial formula for H,,(q,t) was proposed by 
Haglund and Loehr [45], and involved statistics “area” and “dinv.’” These two statis- 
tics are defined on labeled Dyck paths, a geometric encoding of parking functions. 

A Dyck path of order 7 is a lattice path from (0,0) to (n,n) consisting of steps 
East (E = (1,0)) and North (NV = (0,1)), and never going below the line y = x. A 
labeled Dyck path is a Dyck path in which the n north steps are labeled | to n in 
such a way that the labels of consecutive north steps increase from bottom to top. 
See Figure 13.11 for an example, where one places each label in the lattice square to 
the right of the corresponding north step. 
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There is a simple bijection between labeled Dyck paths and parking functions. 
Let P be a labeled Dyck path. For each i € [n], find the north step with label i and 
let a; be the x-coordinate of this step. The fact that P is a Dyck path implies that the 
sequence (a1,...,@,) is a parking function. Conversely, given a parking function, one 
places all cars that prefer the space j in column j + 1 in increasing order, such that 
there is exactly one car in each row, and cars preferring earlier spots appear in lower 
rows of the figure. The labels determine a labeled lattice path in the obvious way, 
which is easily seen to be a Dyck path. Example 13.6.2 shows this correspondence. 


Example 13.6.2 For the parking function a = (2,0,1,0,1,3,6,0), Figure 13.11 
shows the corresponding labeled Dyck path. To see how the labels are assigned, 
take the first column as an example. In a there are three Os, which means that cars 
C2, C4 and Cg prefer space 0. Hence there are three north steps in the Dyck path at 
x =0, with labels 2,4,8 from bottom to top. 


7 
6 
1 

5 

3 
8 
4 
2 


Figure 13.11 
The parking function a = (2,0,1,0,1,3,6,0) and the corresponding labeled Dyck 
path. 


The statistic area(a) of a parking function a is equivalent to the total displacement 
D(a) defined in Section 2. The other statistic, dinv(a), is obtained by counting certain 
inversion pairs between rows of the labeled Dyck path. The exact definitions and 
detailed descriptions of these two statistics are given in the chapter Catalan Paths 
and g,t-enumeration, and hence omitted here. Let 


CHn (q, t) = y goiels) parcala) ; 
acPHy, 


Haglund and Loehr [45] conjectured that CH,,(g,t) = H,(q,t). This conjecture has 
been verified in Maple for n < 11. The truth of the conjecture when g = 1 follows 
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from results of Garsia and Haiman in [35]. It is also the reason that one considers 
parking functions but not other combinatorial objects counted by (n+ 1)”~! here: 
when q = | there is a natural action of G,, on the set Y.%,, that gives the Frobenius 
characteristic of the diagonal harmonics, up to the sign representation. When ¢ = 1, 
the conjecture follows from a result of Loehr and Remmel [62], who proved bijec- 
tively that area and dinv have the same distribution. In addition, Loehr [61] obtained 
a recursion characterizing the polynomial CH,,(q,t), and proved the specialization 


g’"Y?CH,(1/4,q) = (tate 49. 


The joint symmetry of CH,,(q,t), i.e., CHn(q,t) = CH, (t,q), as well as Haglund 
and Loehr’s conjecture are still open. They are part of a larger program on finding 
the combinatorial description for the character of the space of diagonal harmonics 
DH,, which is the same as the character of diagonal coinvariants DR,. It is known 
that the character of DR,, as a doubly-graded G,,-module can be expressed using the 
Frobenius characteristic map as Ve, where e, is the nth elementary symmetric func- 
tion, and V is an operator from the theory of Macdonald polynomials [46]. In 2005 
Haglund, Haiman, Loehr, Remmel, and Ulyanov [44] proposed an explicit combi- 
natorial formula of the expansion of Ve, into monomials. It is usually referred to 
as the “shuffle conjecture” since the formula can be described in terms of statistics 
on certain permutations associated to parking functions, which are shuffles of blocks 
of increasing and decreasing sequences. The shuffle conjecture contains a Garsia- 
Haglund formula for the (q,t)-Catalan number C,,(q,t) [32, 33], the Haglund-Loehr 
conjecture for H,,(q,t), and a formula for (q,t)-Schréder polynomials [42] as special 
cases. 

In the literature on diagonal harmonics, ordinary parking functions enumerated 
by (n+1)""! are called “classical parking functions,” and it is standard to encode 
them as labeled Dyck paths. There are several extensions and generalizations of the 
shuffle conjecture, including an m-parameter extension [44], the rational shuffle 
conjecture [40], and the compositional rational shuffle conjecture [10]. The com- 
binatorial foundations of these extensions are given by rational Dyck paths and ra- 
tional parking functions. Explicitly, let a,b be positive integers. An (a,b)-Dyck path 
is a lattice path from (0,0) to (b,a) consisting of N and E steps, and never going 
below the line y = $x. A rational (a,b)-parking function is an (a,b)-Dyck path to- 
gether with a labeling of the north steps by the set [a] such that labels increase in 
each column from bottom to top. Note that when b = ka for some integer k, the ra- 
tional (a,b)-parking functions are the same as the classical (1,4)-parking functions 
of length a discussed in Section 13.4.4. For other cases, rational parking functions 
are not equivalent to u-parking functions with an arithmetic u. In particular, ratio- 
nal (a,b)-parking functions are counted by b“~! for co-prime positive integers a,b, 
where the ordinary case corresponds to (a,b) = (n,n +1). The combinatorial theory 
of rational Dyck paths and rational parking functions, as well as their relations with 
the representation theory of diagonal coinvariants are discussed in [3]. 
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14.1 Introduction 
14.1.1 Appetizer 


Consider throwing balls labeled 1,2,...,n into a V-shaped bin with perpendicular 


sides. 
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\ @ ay 3) 
Qa 23) Ox0, 
Question 14.1.1 What is the total number of resulting configurations? How many 


configurations are there of any particular shape? 


In order to answer these questions, at least partially, recall the symmetric group 
Sy of all permutations of the numbers 1,...,n. An involution is a permutation 7 € 
Fp, such that 7 is the identity permutation. 


Theorem 14.1.2 The total number of configurations of n balls is equal to the number 
of involutions in the symmetric group -Yn. 


Theorem 14.1.2 may be traced back to Frobenius and Schur. A combinatorial 
proof will be outlined in Section 14.4 (see Corollary 14.4.14). 


Example 14.1.3 There are four configurations on three balls. Indeed, 
{nm © S3: * =1} = {123, 132,213,321}. 


N/M’ NGS 


The inversion number of a permutation 7 is defined by 
inv(z) :=#{i< j : a(i) > a(J)}. 
The left weak order on .%, is defined by 
q<o <> inv(z)+inv(on |) =inv(o). 
The following surprising result was first proved by Stanley [112]. 


Theorem 14.1.4 The number of configurations of (5) balls that completely fill n—1 
levels in the bin is equal to the number of maximal chains in the weak order on Fy. 


The configurations of balls in a bin are called standard Young tableaux. We 
shall survey in this chapter results related to Question 14.1.1 and its refinements. 
Variants and extensions of Theorem 14.1.2 will be described in Section 14.4. Variants 
and extensions of Theorem 14.1.4 will be described in Section 14.11. 
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14.1.2. General 


This chapter is devoted to the enumeration of standard Young tableaux of various 
shapes, both classical and modern, and to closely related topics. Of course, there is 
a limit as to how far afield one can go. We chose to include here, for instance, r- 
tableaux and g-enumeration, but many interesting related topics were left out. Here 
are some of them, with a minimal list of relevant references for the interested reader: 
Semi-standard Young tableaux [69, 114], (reverse) plane partitions [114], solid (3- 
dimensional) standard Young tableaux [25], symplectic and orthogonal tableaux [55, 
20, 127, 12, 128], oscillating tableaux [70, 103, 96, 22, 79], cylindric (and toric) 
tableaux [86]. 


14.2 Preliminaries 
14.2.1 Diagrams and tableaux 


Definition 14.2.1 A diagram is a finite subset D of the two-dimensional integer lat- 
tice Z7. A point c = (i, 7) € D is also called the cell in row i and column j of D; write 
row(c) =i and col(c) = j. Cells are usually drawn as squares with axis-parallel 
sides of length 1, centered at the corresponding lattice points. 


Diagrams will be drawn here according to the “English notation,” by which i enu- 
merates rows and increases downwards, while 7 enumerates columns and increases 
from left to right: 


(1,1)}(1,2)|(1,3) 


(2,1)|(2,2) 


For alternative conventions see Section 14.2.4. 


Definition 14.2.2 Each diagram D has a natural component-wise partial order, in- 
herited from Z?: 
GJ) spf) > tai andj <j. 


As usual, c <p c! means c <p c' butc#c’. 
Definition 14.2.3 Let n := |D|, and consider the set [n] := {1,...,n} with its usual 


linear order. A standard Young tableau (SYT) of shape D is a map T : D — [n| 
which is an order-preserving bijection, namely it satisfies 


cfc SS TO FTC) 


as well as 
Cine ST OST (Ce), 
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Geometrically, a standard Young tableau T is a filling of the n cells of D by the 
numbers 1,...,7 such that each number appears once, and numbers increase in each 
row (as the column index increases) and in each column (as the row index increases). 
Write sh(T) = D. Examples will be given below. 
Let SYT(D) be the set of all standard Young tableaux of shape D, and denote its 
size by 
f? :=|SYT(D)|. 


The evaluation of f? (and some of its refinements) for various diagrams D is the 
main concern of the current chapter. 


14.2.2 Connectedness and convexity 


We now introduce two distinct notions of connectedness for diagrams, and one notion 
of convexity; for another notion of convexity see Observation 14.2.13. 


Definition 14.2.4 Two distinct cells in Z? are adjacent if they share a horizontal or 
vertical side; the cells adjacent to c = (i,j) are (i+1,j) and (i,j +1). A diagram 
D is path-connected if any two cells in it can be connected by a path, which is a 
finite sequence of cells in D such that any two consecutive cells are adjacent. The 
maximal path-connected subsets of a nonempty diagram D are its path-connected 
components. 


For example, the following diagram has two path-connected components: 


Definition 14.2.5 The graph of a diagram D has all the cells of D as vertices, with 
two distinct cells c,c' € D connected by an (undirected) edge if either c <p c' or 
c' <pc. The diagram D is order-connected if its graph is connected. In any case, 
the order-connected components of D are the subsets of D forming connected com- 
ponents of its graph. 


For example, the following diagram (in English notation) is order-connected: 


while the following diagram has two order-connected components, with cells marked 
1 and 2, respectively: 


1|1 


2 


2 


Of course, every path-connected diagram is also order-connected, so that every 
order-connected component is a disjoint union of path-connected components. 
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Observation 14.2.6 If D,,...,D, are the order-connected components of a diagram 


D, then 
|D| ay ak 
po={ p= |)! TT. 
|D1|,---,|Dx| I] Hing 


Definition 14.2.7 A diagram D is line-convex if its intersection with every axis- 
parallel line is either empty or convex, namely if each of its rows {j € Z| (i, j) € D} 
(for i € Z) and columns {i € Z| (i,j) € D} (for j € Z) is either empty or an interval 
[pq] ={p.p+l,.--.q} CZ. 


For example, the following diagram is path-connected but not line-convex: 


14.2.3. Invariance under symmetry 


The number of SYT of shape D is invariant under some of the geometric operations 
(isometries of Z7) that transform D. It is clearly invariant under arbitrary translations 
(i,j) (i+a,j +b). The group of isometries of Z7 that fix a point, say (0,0), is the 
dihedral group of order 8. f? is invariant under a subgroup of order 4. 


Observation 14.2.8 f? is invariant under arbitrary translations of Z?, as well as 
under 

e reflection in a diagonal line: (i, j) > (j,i) or (i,j)  (—j, —i); and 

e reflection in the origin (rotation by 180°): (i,j) 4 (—i, —j). 

Note that f? is not invariant, in general, under reflections in a vertical or hori- 
zontal line ((i, 7) + (i,—j) or (i, 7) (i, j)) or rotations by 90° ((i, j) — (—J, 1) 


or (i, 7) +> (j,—i)). Thus, for example, each of the following diagrams, interpreted 
according to the English convention (see Section 14.2.4), 


has f? =5, whereas each of the following diagrams 


has? = 2. 
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14.2.4 Ordinary, skew and shifted shapes 


The best known and most useful diagrams are, by far, the ordinary ones. They corre- 
spond to partitions. 


Definition 14.2.9 A partition is a weakly decreasing sequence of positive integers: 
A =(Aj,...,A:), where t > Oand A, >... >A, > 0. We say that A is a partition of 
size n = |A| := Y_, A; and length ((A) :=t, and write A | n. The empty partition 
A = () has size and length both equal to zero. 


Definition 14.2.10 Let A = (A,,...,A4;) be a partition. The ordinary (or straight, 
or left-justified, or Young, or Ferrers) diagram of shape A is the set 


D=[A|:={@/|lsistl<j<aij. 
We say that [A] is a diagram of height (A) = t. 


We shall adopt here the “English” convention for drawing diagrams, by which 
row indices increase from top to bottom and column indices increase from left to 
right. For example, in this notation the diagram of shape A = (4,3,1) is 


[A] = (English notation). 


An alternative convention is the “French” one, by which row indices increase from 
bottom to top (and column indices increase from left to right): 


[A] = (French notation). 


Note that the term “Young tableau” itself mixes English and French influences. There 
is also a “Russian” convention, rotated 45°: 


[A] = (Russian notation). 


This notation leads naturally to the “gravitational” setting used to introduce SYT at 
the beginning of Section 14.1. 

A partition A may also be described as an infinite sequence, by adding trailing 
zeros: A; := 0 for i >t. The partition A’ conjugate to A is then defined by 


AV=Hila2 Al (Wi 21). 


The diagram [{A’] is obtained from the diagram [A] by interchanging rows and 
columns. For the example above, A’ = (3,2,2,1) and 


[A] = 
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An ordinary diagram is clearly path-connected and line-convex. If D = [A] is an 
ordinary diagram of shape A we shall sometimes write SYT(A) instead of SYT(D) 
and Fas instead of f?. 


Example 14.2.11 


1/2 
T=(3/4 
7 


Din 


€ SYT(4,3, 1). 


Note that, by Observation 14.2.8, f* = f*’. 


Definition 14.2.12 [fA and wu are partitions such that [p] C [A], namely pj < A; (Vi), 
then the skew diagram of shape A / is the set difference 


D=(A/u)=[A)\ [HI ={G eA) smth si sa} 
of two ordinary shapes. 


For example, 


[(6,4,3,1)/(4,2,1)] = 


A skew diagram is line-convex, but not necessarily path-connected. In fact, its 
path-connected components coincide with its order-connected components. If D = 
[A /] is a skew diagram of shape 2/11 we shall sometimes write SYT(A /{1) instead 
of SYT(D) and f*/# instead of f?. For example, 


1/4 


\oS) 
4 


SYT((6,4,3, 1)/(4,2, 1). 


2 


Skew diagrams have an intrinsic characterization. 

Observation 14.2.13 A diagram D is skew if and only if it is order-convex, namely: 
cc’ ED, JED c<cl<c! Sc ED, 

where < is the natural partial order in Z”, as in Definition 14.2.2. 


Another important class is that of shifted shapes, corresponding to strict parti- 
tions. 


Definition 14.2.14 A partition A = (Ay,...,A;) (t > 0) is strict if the part sizes A; 
are strictly decreasing: A, >... > A, > 0. The shifted diagram of shape A is the set 


D=([A*):={(,|1<i<ti<j<ati-l. 


Note that (Aj +i—1){_, is a weakly decreasing sequence of positive integers. 


Standard Young Tableaux 903 


For example, the shifted diagram of shape A = (4,3,1) is 


[A"] = 


A shifted diagram is always path-connected and line-convex. If D = [A*] is a 
shifted diagram of shape A we shall sometimes write SYT(A*) instead of SYT(D) 
and gt instead of f?. For example, 


1|2|4 
T= (|3\5 
7 


Co 


€ SYT((4,3, 1)*). 


14.2.5 Interpretations 


There are various interpretations of a standard Young tableau, in addition to the inter- 
pretation (in Definition 14.2.3) as a linear extension of a partial order. Some of these 
interpretations play a key role in enumeration. 


14.2.5.1 The Young lattice 


A standard Young tableau of ordinary shape describes a growth process of diagrams 
of ordinary shapes, starting from the empty shape. For example, the tableau T in 
Example 14.2.11 corresponds to the process 


Consider the Young lattice whose elements are all partitions, ordered by inclu- 
sion (of the corresponding diagrams). By the above, a SYT of ordinary shape A is a 
maximal chain, in the Young lattice, from the empty partition to A. The number of 
such maximal chains is therefore f 4 More generally, a SYT of skew shape A/ is a 
maximal chain from p to A in the Young lattice. 

A SYT of shifted shape can be similarly interpreted as a maximal chain in the 
shifted Young lattice, whose elements are strict partitions ordered by inclusion. 


14.2.5.2 Ballot sequences and lattice paths 


Definition 14.2.15 A sequence (a,,...,dy) of positive integers is a ballot sequence, 
or lattice permutation, if for any integers 1<k<nandr>1, 


#{1<i<klaj=r}>#H1 <i<kla;=rt+]}, 


namely: In any initial subsequence (a1,...,a,), the number of entries equal to r is 
not less than the number of entries equal to r +1. 
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A ballot sequence describes the sequence of votes in an election process with several 
candidates (and one ballot), assuming that at any given time candidate | has at least 
as many votes as candidate 2, who has at least as many votes as candidate 3, etc. 
For example, (1,1,2,3,2,1,4,2,3) is a ballot sequence for an election process with 
9 voters and 4 candidates. 

For a partition A of n, denote by BS(A) the set of ballot sequences (a),...,dn) 
with #{i|a; =r} =A, (Vr). 


Observation 14.2.16 The map @ : SYT(A) — BS(A) defined by 
(T)i:=row(T'(i))  (1<i<n) 
is a bijection. 


For example, if T is the SYT in Example 14.2.11 then @(T) = (1, 1,2,2,1,2,3,1). 

Clearly, a ballot sequence a = (a,...,dn) € BS(A) with maximal entry t corre- 
sponds to a lattice path in R’, from the origin 0 to the point A, where in step i of the 
path coordinate a; of the point increases by 1. In fact, BS(A) is in bijection with the 
set of all lattice paths from 0 to A that lie entirely in the cone 


{(x1,..-,%) € R’ |x) >... > x, > OF. 


A SYT of skew shape A /uU corresponds to a lattice path in this cone from pL to A. 
A SYT of shifted shape corresponds to a strict ballot sequence, describing a 
lattice path within the cone 


{(x1,...,%) € RB |x. >... > Xp > X41 =... =X, = O for some 0 < 5 <t}. 


14.2.5.3 The order polytope 


Using the partial order on a diagram D, as in Definition 14.2.2, one can define the 
corresponding order polytope 


P(D):={f:D— [0,lJ|e<pe’ = flo) < f(¢) (Ve, ¢ €D)}. 


The order polytope is a closed convex subset of the unit cube (0, 1]?. Denoting n := 
|D|, each linear extension cy < ... < Cy, of the partial order <p corresponds to a 
simplex 

{f:D— [0,1] | Fler) S---< Flen)} 
of volume 1/n! inside P(D). The union of these simplices is P(D), and this union 
is almost disjoint: Any intersection of two or more simplices is contained in a hy- 


perplane, and thus has volume 0. Each simplex, or linear extension, corresponds to a 
SYT of shape D. Therefore 


Observation 14.2.17 If P(D) is the order polytope of a diagram D then 


fe 
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14.2.5.4 Other interpretations 


The number of SYT of ordinary shape can be interpreted as a coefficient in a power 
series, or as the constant term in a Laurent series; see Remark 14.6.14. 

SYT of certain ordinary, skew and shifted shapes may be interpreted as reduced 
words for suitable elements in the symmetric group. This interpretation will be de- 
veloped and explained in Section 14.11. 

SYT may be interpreted as permutations; ff then measures the size of cer- 
tain subsets of the symmetric group, such as descent classes, sets of involutions, 
Knuth classes and pattern avoidance classes. Examples will be given in Sections 14.4 
and 14.10. 

The Young lattice has a vast generalization to the concept of differential 
posets [113, 42]. 

There are other deep algebraic and geometric interpretations. The interested 
reader is encouraged to start with the beautiful survey [9] and the excellent text- 
books [53, 114, 99, 37, 69]. In the current survey we will focus on combinatorial 
aspects and mostly ignore the algebraic and geometric approaches. 


14.2.6 Miscellanea 


The concepts to be defined here are not directly related to standard Young tableaux, 
but will be used later in this survey. 

A composition of a nonnegative integer n is a sequence (l1,...,[;) of positive 
integers such that UW, +...+; =n. The components pl; are not required to be weakly 
decreasing; in fact, every composition may be re-ordered to form a (unique) partition. 
n= 0 has a unique (empty) composition. 

A permutation o € .%, avoids a pattern 7 € .%; if the sequence (o(1),...,0(m)) 
does not contain a subsequence (o(t),...,0(t,)) (with t) < ... < t,) that is order- 
isomorphic to 7, namely: o(t;) < o(t;) <= m(i) < 2(j). For example, 21354 € 5 
is 312-avoiding, but 52134 is not (since 523 is order-isomorphic to 312). 


14.3. Formulas for thin shapes 
14.3.1 Hook shapes 


A hook shape is an ordinary shape that is the union of one row and one column. For 
example, 


[(6,1°)] = 


One of the simplest enumerative formulas is the following. 
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Observation 14.3.1 For everyn > 1land0<k<n-1, 


(n—k,1F) __ n—1l 
ee) 


Proof. — The letter 1 must be in the corner cell. The SYT is uniquely determined by 
the choice of the other k letters in the first column. | 


Note that, in a hook shape (n + 1 —k,1*), the letter n+ 1 must be in the last cell 
of either the first row or the first column. Thus 


potlkl) oa fork) 4 fk) 


By Observation 14.3.1, this is equivalent to Pascal’s identity 


n n—1 n—1 
= 1<k<n-l). 
eG ea) CA 
Observation 14.3.2 The total number of hook shaped SYT of size n is 2"~'. 


Proof. There is a bijection between hook shaped SYT of size n and subsets of 
{1,...,2} containing 1: Assign to each SYT the set of entries in its first row. 
Alternatively, a hook shaped SYT of size n > 2 is uniquely constructed by adding 
a cell containing n at the end of either the first row or the first column of a hook 
shaped SYT of size n — 1, thus recursively multiplying the number of SYT by 2. 
Of course, the claim also follows from Observation 14.3.1. | 


14.3.2 Two-rowed shapes 


Consider now ordinary shapes with at most two rows. 


Proposition 14.3.3 For everyn > 0 and0<k<n/2 


(n—kk) _ (7) _ n 
pee)! 


where Ce) = 0 by convention. In particular, 


pom) — glmm-l) = 1 eu 


the m-th Catalan number. 


Proof. | We shall outline two proofs, one by induction and one combinatorial. 
For a proof by induction on 7 note first that f (0,0) — if: (1,0) — 7, 
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If 0 < k <n/2 then there are two options for the location of the letter n: at the 
end of the first row or at the end of the second. Hence 


porte = A mea ey a (0 < k < n/2). 


Thus, by the induction hypothesis and Pascal’s identity, 
ip n—1 n—1 n—1 n—1 
sl 8) oS ae) 
n—1 n—1 
= (e') (tea) 
n n 
oa) 


The cases k =0 and k =n/2 are left to the reader. 

For a combinatorial proof, recall (from Section 14.2.5.2) the lattice path inter- 
pretation of a SYT and use André’s reflection trick: A SYT of shape (n — k,k) 
corresponds to a lattice path from (0,0) to (n —k,k) which stays within the cone 
{(x,y) € R*|x > y > 0}, namely does not touch the line y = x + 1. The number of all 
lattice paths from (0,0) to (n—k,k) is (7). If such a path touches the line y= x+ 1, 
reflect its “tail” (starting from the first touch point) in this line to get a path from 
(0,0) to the reflected endpoint (k — 1,n —k +1). The reflection defines a bijection 
between all the “touching” paths to (n —k,k) and all the (necessarily “touching”) 
paths to (k—1,n—k+ 1), whose number is clearly (,”,). | 


Corollary 14.3.4 The total number of SYT of size n and at most 2 rows is (in/2)): 


Proof. — By Proposition 14.3.3, 
[n/2 


ea > ((;) - ts ) : (inal) 


14.3.3 Zigzag shapes 


A zigzag shape is a path-connected skew shape that does not contain a 2 x 2 square. 
For example, every hook shape is zigzag. Here is an example of a zigzag shape of 
size 11: 
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The number of SYT of a specific zigzag shape has an interesting formula, to 
be presented in Section 14.7.1. The total number of SYT of given size and various 
zigzag shapes is given by the following folklore statement, to be refined later (Propo- 
sition 14.10.12). 


Proposition 14.3.5 The total number of zigzag shaped SYT of size n is n\. 


Proof. Define a map from zigzag shaped SYT of size n to permutations in .%, 
by simply listing the entries of the SYT, starting from the southwestern corner and 
moving along the shape. This map is a bijection, since an obvious inverse map builds 
a SYT from a permutation 0 = (01,...,On) € %, by attaching a cell containing 0;+1 
to the right of the cell containing 0; if Oj; > 0;, and above this cell otherwise. | 


14.4 Jeu de taquin and the RS correspondence 
14.4.1 Jeu de taquin 


Jeu de taquin is a very powerful combinatorial algorithm, introduced by 
Schtitzenberger [106]. It provides a unified approach to many enumerative results. 
In general, it transforms a SYT of skew shape into some other SYT of skew shape, 
using a sequence of slides. We shall describe here a version of it, using only forward 
slides, which transforms a SYT of skew shape into a (unique) SYT of ordinary shape. 
Our description follows [99]. 


Definition 14.4.1 Let D be a nonempty diagram of skew shape. An inner corner for 
Disacellc ¢ D such that 


1. DU {c} is a skew shape, and 


2. there exists a cell c' € D such that c < c' (in the natural partial order of Z7, as 
in Definition 14.2.2). 


Example 14.4.2 Here is a skew shape with marked inner corners: 


Here is the main jeu de taquin procedure: 


Algorithm 14.4.3 
Input: T, a SYT of arbitrary skew shape. 
Output: T', a SYT of ordinary shape. 
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1: procedure JDT(T ) 
2 D+ sh(T) 
3 Choose a cell cy = (ig, jo) such that D C (co) + := {(i, j) €Z* :i> io, j > jo} 
4 while co ¢ D do 
5: Choose c = (i,j) € (co) \D which is an inner corner for D 
6 T «+ FORWARDSLIDE(T,c) 
7 D+ sh(T) 
8 end while 
9: return T > Now co € D C (co)+, so D has ordinary shape 
10: end procedure 


And here is the procedure FORWARDSLIDE: 


Algorithm 14.4.4 

Input: (Tin,Cin), where Tin is a SYT of skew shape Din and Cin is an inner corner for 
Din. 

Output: Tou;, a SYT of skew shape Dou = Din VU {cin} \ {c’} for some cl € Din. 


1: procedure FORWARDSLIDE(Tin, Cin) 
2 T & Ty, CH Cin 
3 D+ sh(T) 

4: if c = (i,j) then 
a a+ (i+1,/) 
6 co + (i,j +1) 

7. end if 

8 while at least one of cy and cz is in D do 

C1, ifc, €D but cz €¢D, or cy,c2 € Dand T(c)) < T(c2) 


9: de 
co, ifco €D butc, ¢D, orci,c2 € D and T(c2) < T(c1) 
10: D' —Duf{c}\ {c'} bc¢D,c ED 
I: Define T’ € SYT(D’) by: T’ =T on D\ {c'} and T'(c) :=T(c’) 
12: DeED,TET',cHc 
13: if c = (i, j) then 
14; ca + (it+1,/) 
15: co + (i,j +1) 
16: end if 
17: end while 
18: return T 


19: end procedure 


The JDT algorithm employs certain random choices, but actually the following 
holds. 


Proposition 14.4.5 [106, 130, 131] For any SYT T of skew shape, the resulting SYT 
JDT(T) of ordinary shape is independent of the choices made during the computa- 
tion. 
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Example 14.4.6 Here is an example of a forward slide, with the initial ci, and the 
intermediate cells c marked: 


e/3\6 1/3/6 1/3/6 1/3/6 
Tin = 1|4|7|— e4i7|—> 4|9*|7|— 4\7)@ =Tow 
2|5/8 2|5/8 2|5/8 2|5/8 


and here is an example of a full jeu de taquin (where each step is a forward slide): 


° [3/6 1]3/6] °[1]3|6] [1]3]6 
T= fi[47/> e(47/ ola! - 2l4al7|=T (7). 
2/5/8 2/5[/8 5/8 5/8 


14.4.2 The Robinson-Schensted correspondence 


The Robinson-Schensted (RS) correspondence is a bijection from permutations in 
YF, to pairs of SYT of size n and the same ordinary shape. Its original motivation was 
the study of the distribution of longest increasing subsequences in a permutation. For 
a detailed description see, e.g., the textbooks [114], [99], and [37]. We will use the 
jeu de taquin algorithm to give an alternative description. 


Definition 14.4.7 Denote 6, := [(n,n—1,n—2,...,1)]. For a permutation 1 © .Y, 
let Ty the skew SYT of antidiagonal shape 6,/6,—, in which the entry in the i-th 
column from the left is (i). 


Example 14.4.8 


n=53412 — T= q 


5 


Definition 14.4.9 (The Robinson-Schensted (RS) correspondence) For a permu- 
tation 1 € .F;, let 


Py:=JDT(Iz) and Q,:=JDT(T,-1). 


Example 14.4.10 


3 9) 
HORS ae ] (ee 4 
Tv 4 ’ u 1 
2 3 
Then 
— 773 = 3 °3] «3 
= > > > @/ 1 3/1 > 11/4 
zs as = 214 2 
2 2 2 
= Pr 
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and 
°(2 2 2 2 
e/2] [2 
pak cA Sa ER ee UE ral. eae 
i ell i i 3 3 
3 3 3 3 
T]2]_ 
13 ]4] 7 2s 


Theorem 14.4.11 The RS correspondence is a bijection from all permutations in -, 
to all pairs of SYT of size n and the same ordinary shape. 


Thus 


Claim 14.4.12 For every permutation 1 € -Yy, 
(i) sh(Pr) = sh(Qz). 
(ii) to(P,Q) =a! (Q,P). 
A very fundamental property of the RS correspondence is the following. 


Proposition 14.4.13 [104] The height of sh(z) is equal to the size of the longest 
decreasing subsequence in ™. The width of sh(2) is equal to the size of the longest 
increasing subsequence in 1. 


A version of the RS correspondence for shifted shapes was given, initially, by 
Sagan [100]. An improved algorithm was found, independently, by Worley [135] 
and Sagan [102]. See also [48]. 


14.4.3. Enumerative applications 


In this section we list just a few applications of the above combinatorial algorithms. 
Corollary 14.4.14 


(1) The total number of pairs of SYT of the same shape is n!. Thus 


Yi? =a! 


Xr 


(2) The total number of SYT of size n is equal to the number of involutions in 
-F, [110, A000085]. Thus 


where (2k —1)!:=1-3-...+(2k—1). 
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(3) Furthermore, for every positive integer k, the total number of SYT of height 
<k is equal to the number of [k,k —1,...,1]-avoiding involutions in %,. 


Proof. (1) follows from Claim 14.4.12(i), (2) from Claim 14.4.12(i), and (3) from 
Proposition 14.4.13. a 


A careful examination of the RS correspondence implies the following refine- 
ment of Corollary 14.4.14(2). 


Theorem 14.4.15 The total number of SYT of size n with n—2k odd rows is equal to 
(5,) (2k — 1)!!, the number of involutions in Y, with n— 2k fixed points. 


Corollary 14.4.16 The total number of SYT of size 2n and all rows even is equal to 
(2n—1)!!, the number of fixed point free involutions in Sn. 


For further refinements see, e.g., [120, Ex. 45-46, 85]. 

Recalling the simple formula for the number of two-rowed SYT (Corol- 
lary 14.3.4), it is tempting to look for the total number of SYT of shapes with more 
rows. 


Theorem 14.4.17 [87] The total number of SYT of size n and at most 3 rows is 


n/2l 7, 
M, = y (j)e: 


k=0 


the nth Motzkin number [110, AO01006]. 


Proof. _ By Observation 14.2.6 together with Proposition 14.3.3, the number of SYT 
of skew shape (n—k,k,k)/(k) is equal to 


n 
i Cy ’ 


where C; is kth Catalan number. On the other hand, by careful examination of the 
jeu de taquin algorithm, one can verify that it induces a bijection from the set of all 
SYT of skew shape (n —k,k,k)/(k) to the set of all SYT of shapes (n —k — j,k, j) 
for 0 < j < min(k,n — 2k). Thus 


i [n/2] ere \n/2] 7, 
y fi= y Le APRs) y (5.)e 
k=0 fj =0 


Atn 
e(A)<3 
completing the proof. | 


See [28] for a bijective proof of Theorem 14.4.17 via a map from SYT of height 
at most 3 to Motzkin paths. 

The nth Motzkin number also counts non-crossing involutions in .%,. It follows 
that 
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Corollary 14.4.18 The total number of SYT of height at most 3 is equal to the num- 
ber of non-crossing involutions in Sp. 


Somewhat more complicated formulas have been found for shapes with more 
rows. 


Theorem 14.4.19 [45] 


I. The total number of SYT of size n and at most 4 rows is equal to 
C((n+1)/21Fr41)/21- 


2. The total number of SYT of size n and at most 5 rows is equal to 
|n/2| (2k+2)! 
6d K=0 (34) Ck (k-2) (43)! 

The following shifted analogue of Corollary 14.4.14(1) was proved by 
Schur [105], more than a hundred years ago, in a representation theoretical setting. A 
combinatorial proof, using the shifted RS correspondence, was given by Sagan [100]. 
An improved shifted RS algorithm was found, independently, by Worley [135] and 
Sagan [102]. See the end of Section 14.2.4 for the notation g’. 


Theorem 14.4.20 
y Or) (ph)? =i 


strict AF-n 


14.5 Formulas for classical shapes 


There is an explicit formula for the number of SYT of each classical shape—ordinary, 
skew or shifted. In fact, there are several equivalent fomulas, all unusually ele- 
gant. These formulas, with proofs, will be given in this section. Additional proof 
approaches (mostly for ordinary shapes) will be described in Section 14.6. 


14.5.1 Ordinary shapes 


In this subsection we consider ordinary shapes D = [A], corresponding to partitions 
A. Recall the notation f* := |SYT(A)| for the number of standard Young tableaux 
of shape A. Several explicit formulas are known for this number—a product formula, 
a hook length formula and a determinantal formula. 

Historically, ordinary tableaux were introduced by Young in 1900 [136]. The first 
explicit formula for the number of SYT of ordinary shape was the product formula. 
It was obtained in 1900 by Frobenius [36, eqn. 6] in an algebraic context, as the de- 
gree of an irreducible character x? of .%,. Independently, MacMahon [71, p. 175] 
in 1909 (see also [72, §103]) obtained the same formula for the number of ballot se- 
quences (see Definition 14.2.15 above), which are equinumerous with SYT. In 1927 
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Young [137, pp. 260-261] showed that deg( x") is actually equal to the number of 
SYT of shape /, and also provided his own proof [137, Theorem II] of MacMahon’s 
result. 


Theorem 14.5.1 (Ordinary product formula) For a partition A = (A\,...,A;), let 
O:=A,+t-i (1 <i<t). Then 


fos mas TI 4). 


The best known and most influential of the explicit formulas is doubtlessly the 
Frame-Robinson-Thrall hook length formula, published in 1954 [34]. The story of 
its discovery is quite amazing [99]: Frame was led to conjecture the formula while 
discussing the work of Staal, one of Robinson’s students, during Robinson’s visit to 
him in May 1953. Robinson could not believe, at first, that such a simple formula 
exists, but became convinced after trying some examples, and together they proved 
it. A few days later, Robinson gave a lecture followed by a presentation of the new 
result by Frame. Thrall, who was in the audience, was very surprised because he had 
just proved the same result on the same day! 


Definition 14.5.2 For a cell c = (i,j) € [A] let 
He = [AN ({G AFUE) IF > UTA 7 > B) 
be the corresponding hook, and let 
he := |H,| =Aj;+A;-i-j+1. 
be the corresponding hook length. 


For example, in the following diagram the cells of the hook H(; ») are marked: 


and in the following diagram each cell is labeled by the corresponding hook length: 


6/4/31 
4|2{1 
1 


Theorem 14.5.3 (Ordinary hook length formula) For any partition A = (A1,...,A,), 


a _ [Al 
f Teelaj he 


Last, but not least, is the determinantal formula. Remarkably, it also has a gener- 
alization to the skew case; see the next subsection. 
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Theorem 14.5.4 (Ordinary determinantal formula) For any partition A = (A,,...,A;), 


1 t 
eae 


using the convention 1/k! := 0 for negative integers k. 


Fi =|altedet| 


We shall now show that all these formulas are equivalent. Their validity will then 
follow from a forthcoming proof of Theorem 14.5.6, which is a generalization of 
Theorem 14.5.4. Other proof approaches will be described in Section 14.6. 


Claim 14.5.5 The formulas in Theorems 14.5.1, 14.5.3 and 14.5.4 are equivalent. 


Proof. To prove the equivalence of the product formula (Theorem 14.5.1) and the 
hook length formula (Theorem 14.5.3), it suffices to show that 


Ti-1Ai+t—i)! 
Ne a a oe 
ae Tii,j):i< (Ai -Aj -i+ J) 


This follows by induction on the number of columns, once we show that the product 
of hook lengths for all the cells in the first column of [A] satisfies 


and this readily follows from the obvious 
hei) =A;+t-i (Vi). 


Actually, one also needs to show that the ordinary product formula is valid even 
when the partition A has trailing zeros (so that t in the formula may be larger than 
the number of nonzero parts in A). This is not difficult, since adding one zero part 
A;+1 = 0 (and replacing t by t + 1) amounts, in the product formula, to replacing each 
0; =A; +t—iby 6;,+1 (1 <i<t) and adding ¢,,; =0, which multiplies the RHS of 
the formula by 
| t 
. +1-2 =1. 
To prove equivalence of the product formula (Theorem 14.5.1) and the determi- 
nantal formula (Theorem 14.5.4), it suffices to show that 


1 : 1 
det | ———_—_ =. oer ., 
m acral, T= &! J ( i) 


ipii<j 


where 
rs Ajtt—-i = (1<i<t) 
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as in Theorem 14.5.1. Using the falling factorial notation 


n 


(a)n i= [[(+1-4 (n> 0), 
i=l 
this claim is equivalent to 
det (4) j= [] @-4) 
GAii<i 


which, in turn, is equivalent (under suitable column operations) to the well-known 
evaluation of the Vandermonde determinant 


= [] &@-4). 


Gj)i<j 


Fas 
det ie? 
ij=l 
See [99, pp. 132-133] for an inductive proof avoiding explicit use of the Vander- 
monde. i 


14.5.2 Skew shapes 


The determinantal formula for the number of SYT of an ordinary shape can be ex- 
tended to apply to a general skew shape. The formula is due to Aitken [5, p. 310], and 
was rediscovered by Feit [29]. No product or hook length formula is known in this 
generality (but a product formula for a staircase minus a rectangle has been found 
by DeWitt [21]; see also [61]). Specific classes of skew shapes, such as zigzags and 
strips of constant width, have interesting special formulas; see Section 14.7. 


Theorem 14.5.6 (Skew determinantal formula [5, 29, 114]) The number of SYT 
of skew shape A/, for partitions A = (A,,...,A,) and pb = (Lh,..., bs) with bi < Aj 
(Vi), is 


1 t 
(Ai Mj Te 
with the conventions |; :=0 for j > s and 1/k! :=0 for negative integers k. 


felt = [A lt -det] 


The following proof is inductive. There is another approach that uses the Jacobi- 
Trudi identity. 
Proof. (Adapted from [29].) By induction on the size n := |A /u|. Denote 


1 
aij = ————__: 
7" (aj—pj-its)! 
For n = 0, A; = yy; (Vi). Thus 
(f(a et 


and 
i>j => Ai-—uj-i+j<Ai—pj=A—-Aj <0 => aj =0. 
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Hence the matrix (a;;) is upper triangular with diagonal entries 1, and fel Hols 
0! det(a;;). 

For the induction step assume that the claim holds for all skew shapes of size 
n—1, and consider a shape A/uU of size n with t rows. The cell containing n must be 
the last cell in its row and column. Therefore 


pelt = ply 


where (A/) is the shape 2/~ minus the last cell in row /’, and summation is 
over all the rows i’ that are nonempty and whose last cell is also last in its column. 
Explicitly, summation is over all i’ such that Aj > uy as well as Ay > Ay+ 1. By the 
induction hypothesis, 


fil! =(n-1)! Yet (a‘") 


(?) 


i; 18 the analogue of aj; for the shape (A /j1);; and summation is over the 
above values of i’. In fact, 


@)..— aij, fiz’; 
7 (Ai — Mj —i+ j)- aij, ifi=i’. 


where a 


This holds for all values (positive, zero or negative) of A; — uj —i+j. The rest of the 
proof consists of two steps. 


Step 1: The above formula for fh # holds with summation extending over all 
1 <i’ <t. Indeed, it suffices to show that 
Ay = Uj or Ay = Ai => det (a‘')) =0. 
If Ay = Anat then 


dss Hj—@ ++ i= Ar-D-w-! +i (Wi), 


t 
so that the matrix (a‘t ) 


then 


) has two equal rows and hence its determinant is 0. If Ay = Wy 


isi <i AHI ti <A Mj <A My < Ay — Uy =0 


and 
ji =i (Ay- 1) -py-t +5 < Ay — My S Ay — by =. 
y 
Thus the matrix (a‘i )) has a zero submatrix corresponding to j < i’ < i, which again 
implies that its determinant is zero, for example, by considering the determinant as a 
sum over permutations o € .~ and noting that, by the pigeon hole principle, there is 
no permutation satisfying j = o(i) >i for all i> 7’. 


Step 2: Let A;; be the (i, j)-cofactor of the matrix A = (a;;), so that 


detA =) ajjAij (Vi) 
J 
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and also 
deta =) ajjAij (Vj). 
Then, expanding along row 7’, 
det(ai;) = ay Ay; => Vici = dj)aj jAjj 
J Jj 
where c; := Aj —i and d; := pj; — j. Thus 


file i @) 
(n—1)! ~ d det (a;;°) = VY ler = jay jAv; 
jo an 


U 


Vener Ar; — YY 4547 jij 
J J 


i i! 


= PeydetA—) djdetA = (x ci Ea detA 
i/ j i J 


J 


= (Ea En) detA = |A /p|detA = ndetA 
7 7 


which completes the proof. | 


14.5.3 Shifted shapes 


For a strict partition A, let g* := |SYT(A*)| be the number of standard Young 
tableaux of shifted shape 7. Like ordinary shapes, shifted shapes have three types 
of formulas, namely product, hook length and determinantal. The product formula 
was proved by Schur [105], using representation theory, and then by Thrall [132], 
using recursion and combinatorial arguments. 


Theorem 14.5.7 (Schur’s shifted product formula [105, 132], [69, p. 267, eq. (2)]) 
For any strict partition A = (Ay,...,A4), 


WO. oe 
ee ( A+A; 
Definition 14.5.8 For a cell c = (i, j) € [A*] let 
He =[A*IN((GAIULEA LF > AUG A > FULG+L/)F 2 F413) 


be the correponding shifted hook; note that the last set is relevant only for j <t. Let 


Lj)ii<j 


he :=|H?| = Ait Ajai, re 
I Ae ee yt Sag Sep I |, UP Pee, 


be the corresponding shifted hook length. 
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For example, in the following diagram the cells in the shifted hook Ai >) are 
marked 


and in the following diagram each cell is labeled by the corresponding shifted hook 
length. 


9|7/5|4|2 
4|3 {1 
21 


7 
6 


Theorem 14.5.9 (Shifted hook length formula [69, p. 267, eq. (1)]) For any strict 
partition A = (Ay,...,A1), 


pote! 
Teefaxj he 
Theorem 14.5.10 (Shifted determinantal formula) For any strict partition 1 = 
(Ai,.--,Ar), 
Al! 1 ‘ 
_ A - det | : ; 
Ti,j):i< (Ai +4) (Aj—t+j)! ij=l 

using the convention 1/k! := 0 for negative integers k. 


The formulas in Theorems 14.5.7, 14.5.9 and 14.5.10 can be shown to be equiva- 
lent in much the same way as was done for ordinary shapes in Section 14.5. Note that 
the factors of the first denominator in the determinantal formula (Theorem 14.5.10) 
are precisely the shifted hook lengths h* for cells c = (i, j) in the region j <t. 


14.6 More proofs of the hook length formula 
14.6.1 A probabilistic proof 


Probabilistic proofs rely on procedures for a random choice of an object from a set. 
The key observation is that a uniform distribution implies an exact evaluation and 
“almost uniform” distributions yield good bounds. 

A seminal example is the Greene-Nijenhuis-Wilf probabilistic proof of the or- 
dinary hook length formula, to be described here. Our outline follows Sagan’s de- 
scription, in the first edition of [99], of the original proof of Greene, Nijenhuis and 
Wilf [46]. 

We start with a procedure that generates a random SYT of a given ordinary shape 
D. Recall from Definition 14.5.2 the notions of hook H, and hook length h, corre- 
sponding to a cell c € D. A corner of D is a cell that is last in its row as well as in its 
column (equivalently, has hook length 1). 
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Algorithm 14.6.1 
Input: D, a diagram of ordinary shape. 
Output: A random T € SYT(D). 


1: procedure RANDOMSYT(D) 

2 while D is not empty do 

3: n<|D| 

4: Choose randomly a cell c € D > with uniform probability 1/n 
5 while c is not a corner of D do 

6 Choose randomly a cell c' € H- \ {c} > with uniform probability 


A /(he—1) 


7: ceed 

8: end while 

9: T(c)<-n 
10: D«<D\{c} 
1: end while 
12: return T 


13: end procedure 


We claim that this procedure produces each SYT of shape D with the same prob- 
ability. Explicitly, 


Lemma 14.6.2 The procedure RANDOMSYT produces each SYT of shape D with 
probability 


Proof. By induction on n := |D|. The claim clearly holds for n = 0,1. 

Suppose that the claim holds for all shapes of size n — 1, and let D be an ordinary 
shape of size n. Let T € SYT(D), and assume that T(v) =n for some comer v = 
(a, B). Denote D’ := D\ {v}, and let T’ € SYT(D’) be the restriction of T to D’. 

In order to produce T, the algorithm needs to first produce v (in rows 4-8, given 
D), and then move on to produce 7’ from D’. By the induction hypothesis, it suffices 
to show that the probability that rows 4-8 produce the corner v = (a, B) is 


Tleephe/n! Ay] ee 17 hep 2 he 


Text /=1)l tan aa ig hig g 1? 


where h/, denotes hook length in D’. This is equal to 
> A 1 1 
= 1 as 
tga Oa Sle 


ACla—1] ic¢A 
BC[B— ; 


1 1 


iWijm ; 


Following Sagan [99] we call any possible sequence of cells of D obtained by 
lines 4—8 of the procedure (starting at a random c and ending at the given corner v) a 
trial. For each trial 7, let 


A(t) :={i< a: Ajs.t. (i, 7) is a cell in the trial t} C [1,a—1] 
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be its horizontal projection and let 


B(t):={j < B : dis.t. (i,j) is a cell in the trial t} C [1, B — 1] 


be its vertical projection. 

It then suffices to show that for any given A C [1,a@— 1] and B C [1,8 — 1], 
the sum of probabilities of all trials t ending at v = (@,f) such that A(t) =A and 
B(t) = Bis 


1 1 ll 1 
Nica hip —1 jeB hej —1- 
This may be proved by induction on |A UB]. | 


Lemma 14.6.2 says that the algorithm produces each T € SYT(D) with the same 
probability p. The number of SYT of shape D is therefore 1/p, proving the hook 
length formula (Theorem 14.5.3). 

For a fully detailed proof see [46] or the first edition of [99]. A similar method 
was applied in [101] to prove the hook length formula for shifted shapes (Theo- 
rem 14.5.9 above). 


14.6.2 Bijective proofs 


There are several bijective proofs of the (ordinary) hook length formula. Franzblau 
and Zeilberger [35] gave a bijection that is rather simple to describe, but breaks 
the row-column symmetry of hooks. Remmel [95] used the Garsia-Milne involution 
principle [40] to produce a composition of maps, “bijectivizing” recurrence relations. 
Zeilberger [140] then gave a bijective version of the probabilistic proof of Greene, 
Nijenhuis and Wilf [46] (described in the previous subsection). Krattenthaler [58] 
combined the Hillman-Grassl algorithm [51] and Stanley’s (P, @)-partition theorem 
with the involution principle. Novelli, Pak and Stoyanovskii [76] gave a complete 
proof of a bijective algorithm previously outlined by Pak and Stoyanovskii [80]. A 
generalization of their method was given by Krattenthaler [59]. 

Bijective proofs for the shifted hook length formula were given by Kratten- 
thaler [58] and Fischer [30]. 

A bijective proof of the ordinary determinantal formula was given by Zeil- 
berger [139]; see also [66] and [57]. 

We shall briefly describe here the bijections of Franzblau-Zeilberger and of 
Novelli-Pak-Stoyanovskii. Only the algorithms (for the map in one direction) will 
be specified; the interested reader is referred to the original papers (or to [99]) for 
more complete descriptions and proofs. 

The basic setting for both bijections is the following. 


Definition 14.6.3 Let A be a partition of n and A a set of positive integers such that 
|A| = |A|. A Young tableaux of (ordinary) shape A and image A is a bijection R : 
[A] > A, not required to be order-preserving. A pointer tableau (or hook function) 
of shape A is a function P : |A| > Z that assigns to each cell c € [A] a pointer p(c’) 
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which encodes some cell c' in the hook H, of c (see Definition 14.5.2). The pointer 
corresponding to c’ € H; is defined as follows: 


j,__ ifc’ is j columns to the right of c, in the same row; 
p(c!):=40, if =e; 


—i, ifc’ isi rows below c, in the same column. 


Let YT(A,A) denote the set of all Young tableaux of shape A and image A, PT(A) 
the set of all pointer tableaux of shape A, and SPT(A,A) the set of all pairs (T,P) 
(“standard and pointer tableaux”) where T € SYT(A,A) and P € PT(A). YT(A) is 
a shorthand for YT(A,|n]) where n = |A|, and SPT(A) a shorthand for SPT(A, |n}). 


Example 14.6.4 A typical hook, with each cell marked by its pointer: 


0/1/2{3|4 
—1 
2 


The hook length formula that we want to prove may be written as 


n= f*- J] he. 


ce[A] 


The LHS of this formula is the size of YT(A), while the RHS is the size of SPT(A). 
Any explicit bijection f : YT(A) — SPT(A) will prove the hook length formula. As 
promised, we shall present algorithms for two such bijections. 


The Franzblau-Zeilberger algorithm [35]. The main procedure, that we will call 
FZ-SORTTABLEAU, “sorts” a YT R of ordinary shape, column by column from right 
to left, to produce a SPT (T,P) of the same shape. The pointer tableau P records 
each step of the sorting, keeping just enough information to enable reversal of the 
procedure. Note that @ denotes the empty tableau. 


Algorithm 14.6.5 
Input: RE YT(A). 
Output: (T,P) € SPT(A). 


1: procedure FZ-SORTTABLEAU(R) 
2 (T,P) — (2,2) > Initialize 
3 m <— number of columns of R 
4 for j <m downto | do > Add columns from right to left 
5: c «column j of R 
6 (T,P) <— INSERTCOLUMN(T,P,c) 
7. end for 
8 return (T,P) 
9: end procedure 


The algorithm makes repeated use of the following procedure INSERTCOLUMN: 
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Algorithm 14.6.6 

Input: (T,P,c), where (T,P) € SPT(u,A) for some ordinary shape [and some set 
A of positive numbers of size |A| = || such that all the rows of T are increasing, 
and c = (c1,.--,Cm) is a vector of distinct positive integers c; ¢ A whose length 
m > lM). 

Output: (T’,P’) © SPT(u’,A’), where A' = AU {c1,...,Cm} and wl’ is obtained from 
Lt by attaching a new first column of length m. 


1: procedure INSERTCOLUMN(T, P,c) 


2: fori<1tomdo 
3: T < INSERT(T,i,cj) b> Insert c; into row i of T, keeping the row entries 
increasing 
4: d; < (new column index of c;) — 1 > Initialize the pointer d; 
5: end for 
6: while T is not a Standard Young Tableau do 
7: (k,x) — T~!(min{T (i, j)|T(i—1, j) > T(i,j)}) > The smallest entry 
out of order 
8: yodkitl > Claim: y > 0 
9: T + EXCHANGE(T, (k,x), (k—1,y)) 
10: y’ < new column index of the old T(k—1,y) > The new row index is k 
Ll: > Update the pointers d,_, and dx 
vy, ifd,=v>0, v4Ax-1; 
12: dp <— —l, ifd,=x—1; 
—(ut+1), ifd, = —u <0. 
13: decyl 
14: end while 
15: P+ ATTACH(d,P) > Attach d to Pas a first column 
16: return (T,P) > T is now a SYT 


17: end procedure 


This procedure makes use of some elementary operations, which may be de- 
scribed as follows: 


e INSERT(T,i,c;) inserts the entry c; into row i of T, reordering this row to 
keep it increasing. 


e EXCHANGE(T, (k,x), (€,y)) exchanges the entries in cells (k,x) and (¢,y) of 
T and then reorders rows k and £ to keep them increasing. 


e ATTACH(d,P) attaches the vector d to the pointer tableau P as a new first 


column. 
Example 14.6.7 An instance of INSERTCOLUMN(T,P,c) with 
118 10 e 
T=(|4 , P=|0 » C=B 
7 0 6 
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proceeds as follows (with the smallest entry out of order set in boldface): 


1/8{12) |2 1/8/12) |2 1/4/8) 42 
_ {445 1 3\4 —1 3/12 1 
lara kop eel Vols? Isla «lo 
6 0 6 0 6 0 
1/4/8] 42 
go 0 
5 12 1 
6 0 
and yields 
1|4/8 2} 1/0 
_ [3/7 _ {0/0 
ae 5 12 aes 1/0 
6 0 
An instance of FZ-SORTTABLEAU(R) with 
9 12/8] 1 
_|2|5|4 
i 11/37 
10/6 
proceeds as follows (the second step being the instance above): 
moo fee ep 
Hwr es e " [52 = fifo 
6 0 
1|3|4/8} |O}2)1|0 
2|7|9 2|0/0 = 
* hoi = Eto) 39 =?) 
6 {11 1|0 


The Novelli-Pak-Stoyanovskii algorithm [76]. Again, we prove the hook length 


formula 
ni=f*. TT] he 
ce[A] 
by building an explicit bijection f : YT(A) > SPT(A). However, instead of building 
the tableaux column by column, we shall use a modified jeu de taquin to unscram- 
ble the entries of R € YT(A) so that rows and columns increase. Again, a pointer 
tableau will keep track of the process so as to make it invertible. Our description will 


essentially follow [99]. 
First, define a linear (total) order on the cells of a diagram D of ordinary shape 


by defining 


(i,j) (i,j) ==> either j > j’ or j=’ andi>/’. 
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For example, the cells of the following diagram are labeled | to 7 according to this 
linear order: 


7/4|2 
3} 1 


6 
3 
If RE YT(A) and c € D:= [A], let R<° (respectively R“°) be the tableau consisting 
of all cells b € D with b dc (respectively, <Ic). 

Define a procedure MFORWARDSLIDE, which is the procedure FORWARDSLIDE 
from the description of jeu de taquin, with the following two modifications: 


1. Its input is (T,c) with T € YT rather than T € SYT. 


2. Its output is (T,c) (see there), rather than just T. 


Algorithm 14.6.8 
Input: RE YT(A). 
Output: (T,P) € SPT(A). 


1: procedure NPS(R) 

2 TR 

3 D+ sh(T) 

4: P<0O€PT(D) > A pointer tableau of shape D filled with zeros 

5: while T is not standard do 

6 c € the <-maximal cell such that T° is standard 

7 (T’,c!) + MFORWARDSLIDE(T ““,c) 

8 forb<€Ddo > Replace T~ by T', except that T (c’) < the old T(c) 

T(b), ifboc; 
9: T(b) << T'(b), ifbdcandb4c'; 
T(c), ifb=c. 

10: end for 

LI: Let c = (io, jo) and c! = (ij, j1) > Necessarily ig < i; and jo < ji 
12: for i from ig to i, — 1 do 

13: P(i, jo) < P(it+1,jo)—1 

14: end for 

15: P(it, jo) <— ji — jo 

16: end while 

17: return T > Now co € D C (co)+, so D has ordinary shape 
18: end procedure 
Example 14.6.9 For 

6/2 
R=|4/3] , 
5/1 


here is the sequence of pairs (T,P) produced during the computation of NPS(R) 
(with c in boldface): 


6/2] |0|0 6 0\0 6}1} [0/2 
4|3; |O/O| — |4}1) [OFl — |[4{2|} [0)0 
S}1} [0|0 5/3} [0/0 5/3} [0/0 
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61} |O}2 6}1} |O}2 1/4} [042 
—> |4/)2] |O;0; — {|2/4) {1{0;| > [2/5] [0j0}. 
3/5] [1|0 3/5] [10 3}6| (|1|0 


14.6.3 Partial difference operators 


MacMahon [72] has originally used partial difference equations, also known as recur- 
rence relations, to solve various enumeration problems, among them the enumeration 
of ballot sequences, or equivalently SYT of an ordinary shape (see Section 14.2.5.2). 
Zeilberger [138] improved on MacMahon’s proof by extending the domain of defini- 
tion of the enumerating functions, thus simplifying the boundary conditions: In PDE 
terminology, a Neumann boundary condition (zero normal derivatives) was replaced 
by a Dirichlet boundary condition (zero function values). He also made explicit use 
of the algebra of partial difference operators; we shall present here a variant of his 
approach. 

Consider, for example, the two dimensional ballot problem, namely finding the 
number F(mj,,m2) of lattice paths from (0,0) to (m,,m2) that stay in the region 
my, > my > 0. MacMahon [72, p. 127] has set the partial difference equation 


F(m,m2) = F(m, — 1,mz) + F(m,m2 — 1) (m, > m2 > 0) 
with the boundary conditions 
F(m,,mz) = F(m,m2—1) (m, =m > 0) 


and 
F(m,,0) =1 (m, > 0). 


By extending F to the region m, > mz — 1, the recursion can be required to hold for 
almost all m,; > m2 > 0: 


F(m,,m2) = F(m, — 1,mz) + F(m,,m2 — 1) (m, > m2 > 0, (m1,m2) 4 (0,0)) 
with 
F(m,m2) =0 (m, =m2—1 or m =-—1) 


and 
F(0,0) = 1. 


In general, consider functions f : Z” — C and define the fundamental shift operators 
X1,---,Xn by 


(Xif)(m1,.--,1Mn) = f(mm,...,m;+1,...,7n) dd S15 n): 


For a € Z” write X* = X/"'---X,, so that (Xf) (m) = f(m+@). A typical linear 
partial difference operator with constant coefficients has the form 


Pap i ae 


a>0 
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for some polynomial p(z) with complex coefficients, so that dg € C for each a = 
(Q1,---,; On) € N” (where N is the set of nonnegative integers) and aq 4 0 for only 
finitely many values of @. We also assume that p(0) = ap = 1. 


Definition 14.6.10 Define the discrete delta function 6 : Z” — C by 


1, ifm=0; 
aim) = {4 if 


0, otherwise. 


A function f : Z" + C satisfying Pf = 6 is called a fundamental solution corre- 
sponding to the operator P. If f is supported in N", it is called a canonical funda- 
mental solution. 


It is clear that each operator P as above has a unique canonical fundamental solution. 
In the following theorem we consider a slightly more general type of operators, 
which can be written as X% p(X, ',...,X,,!) for a polynomial p and some a > 0. 


Theorem 14.6.11 (A variation on [138, Theorem 2]) Let F, = F,,(m,,...,1n) be 
the canonical fundamental solution corresponding to an operator P = px ee Soa ), 
where p(z) is a symmetric polynomial with p(0) = 1. Denote 


Ay:= [] (f-%:X;'). 


GA)i<j 
Then G,, = A,F;, is the unique solution of the equation Pg = 0 in the region 
{(m,...,mn) € Z”|m, >... > my, > OF \ {(0,...,0)} 


subject to the boundary conditions 


(si) mj = mj, —1 = g(m,...,1n) =0, 


My = —1 => g(m,...,mn,) =0 


and 


Proof. Since each X; commutes with the operator P, so does A,. Since m, +...+my 
is invariant under X;X nS and F,, is a solution of Pf = 0 in the complement of the 
hyperplane m,; +...+-m, = 0, so is G,. It remains to verify that G,, satisfies the 
prescribed boundary conditions. Now, by definition, 


G,(m) = (I— X1Xy')A12F,(m) 


where the operator Aj,2 is symmetric with respect to X;, X2. Since Fy, (m) is a sym- 
metric function, we can write 


G,(m) = (I—XiX;')H(m) 
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where H is symmetric with respect to m; and m2. Suppressing the dependence on 
M3,--.,Mn, 

G,(m1,m2) = (I —X1X5')H(m1,m) => H(m,,m2) —H(m + l,m) — 1) =0 


whenever m, = mz — 1, by the symmetry of H. Similarly, fori =1,...,2—1, G,(m) = 
0 on m =m, — 1. Onm, = —1, 


Gr(m,...,1!m—1,—1) = I] 3 @-%ix7") 
(i,j):1<i<j<n-1 


[Tl @-X:X7')Fa(m,...,7m-1,-1). 


1<i<n-1 
Since F,,(m) = 0 for m, < 0, 
G,(m,...,;Mn—1,—-1) =0. 
Finally, F,(7,...,™,) = 0 on all of the hyperplane m, + ...++my, = 0 except the 
origin 0 = (0,...,0). Therefore 
Gn(0) = AnFn(0) = Fn(0) = 1. : 


For every function f : Z” — C whose support is contained in a translate of N” 
(i.e., such that there exists N € Z such that f(m,...,m,) = 0 whenever m; < N for 
some i) there is a corresponding generating function (formal Laurent series) 


gt(f) = i f(m)z™ € C((z1,.--,Zn))- 


Let p(z1,.--,;Zn) be a polynomial with complex coefficients and p(0) = 1, and let 
P= p(X, ! ,...,X,!) be the corresponding operator. Since gf(X~*f) = z* gf(f) we 
have 

gf(Pf) = p(z,---.2n)-gf(f), 
and therefore Pf = 6 implies gf(f) = 1/p(z1,...,2n). 
Definition 14.6.12 (MacMahon [72]) Let A C Z” and f :A — C. A formal Laurent 


series Y.,,a(m)z” is a redundant generating function for f on A if f(m) = a(m) 
forallmeéA. 


Theorem 14.6.13 (MacMahon [72, p. 133]) Let g(m) be the number of lattice 
paths from 0 to m, where travel is restricted to the region 


A={(m,...,Mn) € Z"|m, > m2 >... > my > OF. 


I] (1 21) l 
(i,j)i<j Si 1—z-—...-Zn 


is a redundant generating function for g on A and therefore 


g(m) = [] @i-mj+i-i. 


(ij)i<j 


Then 


(my +...+m,)! 
(m,+n—1)!---m,! 
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This gives, of course, the ordinary product formula (Theorem 14.5.1). 


Proof. Apply Theorem 14.6.11 with P = J —x;! —...—X7!. The canonical fun- 
damental solution of Pf = 6 is easily seen to be the multinomial coefficient 


m!my! ? 


(m+-+7)! if m. > 0 (Vi): 
Fatma) =| if m; > 0(Vi); 


0, otherwise, 
with generating function (1 —zj —...—Zn)7!. 
The number of lattice paths in the statement of Theorem 14.6.13 clearly satisfies 
the conditions on g in Theorem 14.6.11, and therefore 


g=G,=AiFr (on A). 


This implies the claimed redundant generating function for g on A. 
To get an explicit expression for g(m) note that (m; +...-++m,)! is invariant under 
XiX;", so that 


I 


g(my,...,Mn) 


(i, j)i<j 


= (m,+...+m,)!- IT a—xxy) |]. 


Il (1-xx;') eed 


m!++-my! 


Consider 


(i,j):i 
I] &;'- 
(x; 


a 

> 
x 

Q 


ijji<j i 


mt Lima 
aah, bo) Neate |” 


where ¢; := mj+n—i(1 <i <n). Clearly H is an alternating (anti-symmetric) func- 
tion of ¢),...,€n, which means that 


xe) TE Ee ] 
x") 


QO(41, £2,.--,&n) 


NM) = gon bil 


where Q is an alternating polynomial of degree n — | in each of its variables. g, and 
therefore also H and Q, vanish on each of the hyperplanes m; = mj+; — 1, namely 
€; = lin, (I <i <n—1). Hence ¢; — ¢;,1, and by symmetry also ¢; — ¢; for each 


i # j, divide Q. Hence 
a(e)=c [] -4)) 


Gji<j 
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and 
my+...+m,)! * tf 
g(m) =c ; ; [] (m-—m;-i+J) 
(my +n—1)!-+-m! Gpsi<i 
for a suitable constant c, which is easily found to be 1 by evaluating g(0). a 


Remark 14.6.14 Theorem 14.6.13 gives an expression of the number of SYT of 
ordinary shape A = (A;,...,A;) as the coefficient of z’ (where £; = A; +t — i) in the 
power series 


1 
I] (zi 2) ‘ 1— = = ae? 
(i.f:i<i ein ve ee 
or as the constant term in the Laurent series 
ag; 1 
Zz Zizi ye : 
Tz I] (zi i) 1—z—...-% 


i (G.jri<j 


14.7 Formulas for skew strips 


We focus our attention now on two important families of skew shapes, which are of 
special interest: Zigzag shapes and skew strips of constant width. 


14.7.1 Zigzag shapes 


Recall (from Section 14.3.3) that a zigzag shape is a path-connected skew shape that 
does not contain a 2 x 2 square. 


Definition 14.7.1 For any subset S C [n— 1] := {1,...,n—1} define a zigzag shape 
D = zigzag,,(S), with cells labeled 1,...,n, as follows: Start with an initial cell la- 
beled |. For each 1 <i<n-—1, given the cell labeled i, add an adjacent cell labeled 
i+1 above celli ifi € S, and to the right of cell i otherwise. 


Example 14.7.2 


Wal 


n=9,S={1,3,5,6} — — _zigzago(S) = 


vo] 


1 


This defines a bijection between the set of all subsets of [n — 1] and the set of all 
zigzag shapes of size n (up to translation). The set S consists of the labels of all the 
cells in the shape zigzag,,(S) such that the cell directly above is also in the shape. 
These are exactly the last (rightmost) cells in all the rows except the top row. 
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Recording the lengths of all the rows in the zigzag shape, from bottom up, it fol- 
lows that zigzag shapes of size n are also in bijection with all the compositions of n. 
In fact, given a subset S = {51,...,5,} C [n—1] (with 5) < ... < sx), the composition 
corresponding to zigzag,,(S) is simply (1,52 —51,..-,8% — 5-1, — Sx). 


Theorem 14.7.3 [72, 119] Let S = {s1,...,se} C [n—1] (51 <... <x) and set sp := 
0 and sx44 :=n. Then 


k k 
f7e78n(S) — n!-det feed =aet| ( ie )| =aet| ( oe ue 
(sj44 — si)! i,j=0 Sj+1— Si i,j=0 Sj+1 — Si 


For example, the zigzag shape 


[A/E] = 


corresponds to n = 9 and S = {2,6}, and therefore 


a a oO (2) @) 6) () () 6) 
pem=9edet} 1 fe] =4et] (2) C) GP =4t}G) @ G) 
a 0 ) G 0 6) G) 


Theorem 14.7.3 is a special case of the determinantal formula for skew shapes (The- 
orem 14.5.6). We shall now consider a specific family of examples. 


Example 14.7.4 Consider, for each nonnegative integer n, one special zigzag shape 
D,. For n even it has all rows of length 2: 


and for n odd it has a top row of length | and all others of length 2: 


Clearly, by Definition 14.7.1, D, = zigzag, (S) for S = {2,4,6,...} C [n—1]. 
Definition 14.7.5 A permutation o © S;, is up-down (or alternating, or zigzag) if 


o(1) < o(2) > o(3) < o(4) >... 
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Observation 14.7.6 If we label the cells of Dj, as in Definition 14.7.1 (see Exam- 
ple 14.7.2), then clearly each standard Young tableau T : Dy, — |n| becomes an up- 
down permutation, and vice versa. (For an extension of this phenomenon see Propo- 
sition 14.10.12.) 


Up-down permutations were already studied by André [6, 7] in the nineteenth cen- 
tury. He showed that their number A,, [110, AO00111] satisfies 


Proposition 14.7.7 


= secx + tanx. 
n! 


They are therefore directly related to the secant (or zig, or Euler) numbers E£,, [1 10, 
A000364], the tangent (or zag) numbers 7,, [110, A000182] and the Bernoulli 
numbers B,, [110, A000367 and A002445] by 


Aan = (—1)"Eon (n = 0) 
and 
(1) 24" _ 1) 
2n 


Note that there is an alternative convention for denoting Euler numbers, by which 
E,, = Ap for all n. 


Ao—1 = Th = Bon (n> 1). 


Proposition 14.7.8 


etn 
n=) ({)ArAn-t (n> 1) 
with Ag = Aj = 1. 


Proof. InaSYT of the required shape and size n+ 1, the cell containing n+ 1 
must be the last in its row and column. Removing this cell leaves at most two path- 
connected components, with the western/southern one necessarily of odd size (for 
n> 1). It follows that 


Applying a similar argument to the cell containing | gives 


n 


n 
Ania): (j,)ArAn-t (n> 0), 


and adding the two formulas gives the required recursion. | 
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Indeed, the recursion for A,, (Proposition 14.7.8) can be seen to be equivalent to 
the generating function (Proposition 14.7.7) since f(x) = secx + tanx satisfies the 
differential equation 


2f (x) = 14 f(x) 


with f(0) = 1. 
Proposition 14.7.3 thus gives the determinantal formulas 
a2 it 1 1 1 1 
2 4! OG! (2n—4)!  (2n—2)! — (2n)! 
124 1 1 
21 - (2n—4)!  (2n—2)! 
(—1)"Ex, =(2n)!-det} 9 1 x On—ZyI 
OF? OOM OE nw 1 ii 
and 
1 41 1 1 1 
2 4 6! (2n—4)!  (2n—2)! 9 (nT)! 
1 2 4 ! 
2 4! (2n—4)!  (2n—3)! 
T,=(2n—1)!-det} 9 1 F -  Baesyl 
0 0 O 1 i 


14.7.2 Skew strips of constant width 
The basic skew strip of width m and height n is the skew diagram 
Dan = ((nt+m—1,n+m—2,...,m+1,m)/(n—1,n—2,...,1,0)]. 


It has 1 rows, of length m each, with each row shifted one cell to the left with respect 
to the row just above it. For example, Dz, is D2, from Example 14.7.4 above while 


D355 = 


The general skew strip of width m and height n (m-strip, for short), Dyn... has 
arbitrary partitions A and yw, each of height at most k := |m/2], as “head” (north- 
eastern tip) and “tail” (southwestern tip), respectively, instead of the basic partitions 
A =u =(k,k—1,...,1). For example, 


ee e,/e\|e 
e\|e 
e 
Do7,(4,2,1),(3,3,1) = 
e 
e|;e;\e 
e,;e\e 


where m = 6, n = 7, k = 3 and the head and tail have marked cells. 
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The determinantal formula for skew shapes (Theorem 14.5.6) expresses f? as an 
explicit determinant of order n, the number of rows. Baryshnikov and Romik [11], 
developing an idea of Elkies [27], gave an alternative determinant of order k, half the 
length of a typical row. This is a considerable improvement if m, k, A and pL are fixed 
while n is allowed to grow. 

The general statement needs a bit of notation. Denote, for a non-negative integer 
n, 

An ", A 


a (2"—1)A;, 
nl? om gnt+l 1? 


Qn 2 
where A,, are André’s numbers (as in the previous subsection); and let 


Oe ke en 
0, if n is odd. 


A:= 


n 


mM 
A, i= 


Define, for nonnegative integers N, p and q, 


X (p,q) := Ret EE (1) Ange ai + e(p+1) ae trae 
poe i=0 j=0 (p—2i)\(q—2j)! j=0 (q—-2j)! 
LI] (—-1)'AN voit 

Peed) y ace ie +e(ptle(qt+ DAnyp+q+t 
i=0 : 
and 
xX O(p q) = y (CUTAN 244 + eé(p) v (IVAN psa 
pic, i=0 j—0 (p —2i)\(q—2j)! j=0 (q-2))! 


Le/2] (-1)'AY 
N+2i+q+1 m 
ee eee E(qg)A 
=| (p = 2i)! + (p) (q) N+p+q+1 
Theorem 14.7.9 [11, Theorem 4] Let D= Dyn, be an m-strip with head and tail 
partitions A = (Ay,...,A,) and U = (M1,..., Ux), where k:= |m/2|. For) <i<k 
define L; := A; +k —i and M; := pj +k—i, and denote 


m%2 = 0, ifmis even; 
1, ifmis odd. 


+€(q) 


Then 
k 


k 
f? =(—1)@|D|t-det [x3 ier (LinM)) 
ij= 

Note that x) (p,q) are linear combinations of Ay+1,...,An+p+q+1, So that f? 
is expressed as a polynomial in the A; whose complexity depends on the row length 

m but not on the number of rows n. 
The impressive formal definitions of X, ) have simple geometric interpretations: 

0) 


D 
( f 
Xpn-1 (P+) = DI! 
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where 


D = Zigzagony pig({P+2,p+4,...,pt+2n—2}) = 


(p marked southwestern cells in a row, 27 unmarked cells, and g marked northeastern 
cells in a row), and 


D 
(0) Pe 
Xan (P14) = 
||! 
where 
D = 2igzagons pgp {PD +2,p+4,...,p+2n,pt+2n+1,...,p+2n+q}) 


(p marked southwestern cells in a row, 2n + 1 unmarked cells, and g marked north- 
eastern cells in a column). These are 2-strips, i.e. zigzag shapes. It is possible to 
define x) (p,q) similarly in terms of 3-strips, a task left as an exercise to the 
reader [11]. 

Here are some interesting special cases. 


Corollary 14.7.10 [11, Theorem 1] 3-strips: 


f P3000 = _Bn= 2) Tr 
(2n — 1)!22n-2° 
fP3%).0 = _Bna=WTn_ 
(2n — 1)!22n-1” 
fP3n = fP3n(1).(1) =, (3n)! (277 dos DT 


(2n— 1)!22"-1(22" — 1) 


Corollary 14.7.11 [11, Theorem 2] 4-strips: 


2 
fP4n0.0 = (An —2)! Tr, Ean-2E an 
(2n—1)2 ' (2n—2)!(2n)! 


Ee. Eon2E 
Dan — ¢Pan(1).(1 4, 2n 2n—-2L.2n+2 
Ce =n (az (2n—2)!(2n +2)! 
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Corollary 14.7.12 [11, Theorem 3] 5-strip: 


(5n—6)!T? , 
(2n — 3)!224n-6(22n—2 _ J)’ 


fP 500 = 


Proof of Theorem 14.7.9 (sketch). |The proof uses transfer operators, following 
Elkies [27]. Elkies considered, essentially, the zigzag shapes (2-strips) D, from Ex- 
ample 14.7.4, whose SYT correspond to alternating (up-down) permutations. Recall 
from Section 14.2.5.3 the definition of the order polytope P(D,,), whose volume is, 
by Observation 14.2.17, 

tis 


vol P(D,) = ze 
n! 


This polytope can be written as 
P(Dn) = {(%1,--- Xn) € (0, 1]" : x1 <2 S03 <g>... }, 


and therefore its volume can also be computed by an iterated integral: 


1 1 x2 1 
vol P(D,) =| dx; | dsp | dvs | dx4:--- . 
1 3 


Some manipulations now lead to the expression 
vol P(D,,) = (T"~! (1),1) 


where 1 € L”/0, 1] is the function with constant value 1, (-,-) is the standard inner 
product on L7{0,1], and T : L?[0,1] + L7[0,1] is the compact self-adjoint operator 
defined by 


rpw=["soydy wre x200,1). 


The eigenvalues A, and corresponding orthonormal eigenfunctions @, of T can be 
computed explicitly, leading to the explicit formula 


vol P(D,) = LAM Oe)” > qt! ay (4k + 1)r+1 (n = 1) 


which gives a corresponding expression for 
An = f?" =n! volP(Dn). 


Baryshnikov and Romik extended this treatment of a 2-strip to general m-strips, using 
additional ingredients. For instance, the iterated integral for a 3-strip 
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gives 
voIP(D3,,4,) = ((BA)"~!Ty (1), 7 (1) 


where Q := (u ,v) € [0,1]? : w< v}, the transfer operators A : L7[0, 1] + L?(Q) and 
B: L?(Q) — L’(0,1] are defined by 


(Afar): [Flax 


x pl 
=a / g(u,v)dvdu, 
0 Jx 


and 7,, T, are operators corresponding to the “head” and “tail” partitions A and py. 


and 


Ina slightly different direction, Stanley [118] defines Depen to be the skew shape 
whose diagram has n rows, with a cells in the top row and b cells in each other 
row, and such that each row is shifted c — 1 columns to the left with respect to the 
preceding (higher) row. For example, 


Ds543,4 = 


Theorem 14.7.13 [118, Corollary 2.5] For a, b and c with c <b < 2c, 


n 


yt (=x) 
> qe oa ; *Yn>0 (b+nc)! 
(a+nb)! 


n>0 


~ =e) 2350 
Two special cases deserve special attention: a= b and b=c. 
For a = b all the rows are of the same length. 

Corollary 14.7.14 For a and c with c <a < 2c, 


4 
LAE ven = (1 x Y (—x) ? 


sai a-—c)! 54 (atne)! 


In particular, fora = b =3 andc =2, D332. = D3, as in Theorem 14.7.10: 


Ei 2n -(5 (—x7)" )-2 
(3n)! (2n+1)! sinx 


n>0 n>0 


This result was already known to Gessel and Viennot [43]. 
For b =c one obtains a zigzag shape: Dacen+1 = Zigzag,4,-(S) for S = 
{c,2c,...,nc}. 


Corollary 14.7.15 For any positive a and c, 
(=x)" 
L (fB8t8a+ne ({c,2c,...,nc}) ii _ *Yn>07 (c+nc)! 


z Fm Tao Ey 
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14.8 Truncated and other non-classical shapes 


Definition 14.8.1 A diagram of truncated shape is a line-convex diagram obtained 
from a diagram of ordinary or shifted shape by deleting cells from the northeastern 
corner (in the English notation, where row lengths decrease from top to bottom). 


For example, here are diagrams of a truncated ordinary shape 


[(4,4,2, 1) \ (1)] "a 


and a truncated shifted shape: 


[(453,2, VG 1)] = 


Modules associated to truncated shapes were introduced and studied in [54, 94]. 
Interest in the enumeration of SYT of truncated shapes was recently enhanced by 
a new interpretation [4]: The number of geodesics between distinguished pairs of 
antipodes in the flip graph of inner-triangle-free triangulations is twice the number 
of SYT of a corresponding truncated shifted staircase shape. Motivated by this re- 
sult, extensive computations were carried out for the number of SYT of these and 
other truncated shapes. It was found that, in some cases, these numbers are unusually 
“smooth,” i.e., all their prime factors are relatively very small. This makes it reason- 
able to expect a product formula. Subsequently, such formulas were conjectured and 
proved for rectangular and shifted staircase shapes truncated by a square, or nearly a 
square, and for rectangular shapes truncated by a staircase; see [1, 81, 125, 123]. 


14.8.1 Truncated shifted staircase shape 


In this subsection, A = (Ay,...,A,) (with A; >... > A; > 0) will be a strict partition, 
with gt denoting the number of SYT of shifted shape /. 

For any nonnegative integer n, let 6, := (n,n —1,...,1) be the corresponding 
shifted staircase shape. By Schur’s formula for shifted shapes (Theorem 14.5.7), 


Corollary 14.8.2 The number of SYT of shifted staircase shape 6, is 


n—1 i! 


iets 
2 Lien’ 


where N := |6,| = a: 
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The following enumeration problem was actually the original motivation for the 
study of truncated shapes, because of its combinatorial interpretation, as explained 
in [4]. 


Theorem 14.8.3 [1, 81] The number of SYT of truncated shifted staircase shape 
On \ (1) is equal to 
Sn CrCn—2 
2Con-3. 


2n 


og) is the n-th Catalan number. 


where Cy, = 4 ( 


Example 14.8.4 There are g°4 = 12 SYT of shape 5y, but only 4 SYT of truncated 
shape 54 \ (1): 


1|2/3 1/2/4 1|2/3 1/2|4 
4|5/|6 3|5|6 4|5|7 3|5|7 
7/8) ? 7/8) 6|8] ° 6/8 
9 9 9 9 


Theorem 14.8.3 may be generalized to a truncation of a (k— 1) x (k — 1) square 
from the northeastern corner of a shifted staircase shape 6,424. 


Example 14.8.5 For m= 1 and k = 3, the truncated shape is 


[55 \ (2*)] = 


Theorem 14.8.6 [1, Corollary 4.8] The number of SYT of truncated shifted staircase 
shape bn+2% \ ((k—1)~") is 


glerkt lcm suet heal) glmtetl sits mt) ; N!M! 
(N—-M—1)!2M+1)!’ 


where N = ae) —(k—1)? is the size of the shape and M = k(2m+k+3)/2-1. 


Similar results were obtained in [1] for truncation by “almost squares,” namely 
by (kk =1). 


14.8.2 Truncated rectangular shapes 


In this section, A = (A1,...,Am) (with A; >... > Am > 0) will be a partition with 
(at most) m parts. Two partitions that differ only in trailing zeros will be considered 
equal. 

For any nonnegative integers m and n, let (n’”) := (n,...,n) (m times) be the 
corresponding rectangular shape. The Frobenius-Young formula (Theorem 14.5.1) 
implies the following. 
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Observation 14.8.7 The number of SYT of rectangular shape (n") is 


where 


Consider truncating a (k — 1) x (k— 1) square from the northeastern comer of a 
rectangular shape ((n + k—1)"**!). 


Example 14.8.8 Let n = 3, m=2 and k =3. Then 


[(5*)\ (27) = 


Theorem 14.8.9 [1, Corollary 5.7] The number of SYT of truncated rectangular 
shape ((n-+k—1)™**-!)\ ((k—1)*!) is 
N\(mk —1)\(nk—1)'(m+n—1)IK Fy Fy—-1Fe-1 
(mk + nk — 1)! Fintn+k-1 


where N is the size of the shape and Fy, is as in Observation 14.8.7. 
In particular, 


Corollary 14.8.10 The number of SYT of truncated rectangular shape ((n+1)"*')\ 


(1) is 
N\(2m—1)\(2n—1)!-2 Fp—1Fy—1 


(2m + 2n — 1)!(m-+n) Fintn 
where N = (m+1)(n+1)—1 is the size of the shape and F, is as in Observa- 
tion 14.8.7. 


2 


Similar results were obtained in [1, 81] for truncation by almost squares kK = 
(Ke-" k=). 

Not much is known for truncation by rectangles. The following formula was con- 
jectured in [1] and proved by Sun [124] using Selberg-type integrals. 


Proposition 14.8.11 [124] Forn > 2 


fO\2) = _(P—2IGn—AP-6 Fa 
(6 —8)!(2n—2)\(n—2)!? Fan-4 


where F,, is as in Observation 14.8.7. 


The following result was proved by Snow [111]. 
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Proposition 14.8.12 [111] Forn >2 andk>0 


(kn—k)\(kn+n)! FeFn 
(knt+n—k)! Fark’ 


foerNr-2) 


where F, is as in Observation 14.8.7. 


A different method to derive product formulas, for other families of truncated 
shapes, has been developed by Panova [81]. Consider a rectangular shape truncated 
by a staircase shape. 


Example 14.8.13 


[(4°) \ &] = 


Theorem 14.8.14 [81, Theorem 2] Let m >n > k be positive integers. The number 
of SYT of truncated shape (n) \ 5, is 


N Fa k 1)” gman 1,...,m ~)E(kK+1,m,n—k—1) 
m(n—k—1) E(k+1,m,0)  ” 


where N = mn— Ae is the size of the shape and 


1 ll 1 
(1428)? 5 jc, (+28) (2p—1+25+2)) UPI? 


Il ifr is even; 
E(r,p,s) = 2 releapart2 


r—1)/2+s)! ast 
ES E(r 15 p85 ifr is odd. 


14.8.3 Other truncated shapes 


The following elegant result regarding shifted strips was recently proved by Sun. 


Theorem 14.8.15 [126, 84.2] The number of SYT of truncated shifted shape with n 
rows and 4 cells in each row 


is the (2n — 1)th Pell number [110, A000129] 


A (a4-y2)"= a=v2"2). 
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Sun applied a probabilistic version of computations of volumes of order poly- 
topes to enumerate SYT of truncated and other exotic shapes. In [123] he obtained 
product formulas for the number of SYT of certain truncated skew shapes. This in- 
cludes the shape ((n+k)’*!,n"~!)/(n— 1)" truncated by a rectangle or an “almost 
rectangle,” the truncated shape ((n + 1)?,n”~*) /(n — 2) \ (27), and the truncated 
shape (n+1)?/n™~? \ (2). 

Modules associated with non-line-convex shapes were considered in [54]. The 
enumeration of SYT of such shapes is a very recent subject of study. Special non- 
line-convex shapes with one box removed at the end or middle of a row were consid- 
ered in [125]. For example, 


Proposition 14.8.16 [125, Theorem 5.2] For m > 0, the number of SYT of shape 
(m+ 3,3,3) with middle box in the second row removed, is 


m+5 (m+2\ (/m+9 
10 2 2, of 
There are very few known results in this direction; problems in this area are wide 
open. 


14.8.4 Proof approaches for truncated shapes 


Different approaches were applied to prove the above product formulas. We will 
sketch one method and remark on another. 

The pivoting approach of [1] is based on a combination of two different bijec- 
tions from SYT to pairs of smaller SYT: 


1. Choose a pivot cell C in the northeastern boundary of a truncated shape ¢ and 
subdivide the entries of a given SYT T into those that are less than the entry 
T (c) and those that are greater. 


2. Choose a letter t and subdivide the entries in a SYT T into those that are less 
than or equal to ¢ and those that are greater than f. 


Proofs are obtained by combining applications of the first bijection to truncated 
shapes and the second bijection to corresponding non-truncated ones. Here is a typi- 
cal example. 


Proof of Theorem 14.8.3 (sketch). First, apply the second bijection to a SYT of a 
shifted staircase shape. 

Let n and t be nonnegative integers, with tf < Cans Let T be a SYT of shifted 
staircase shape 6,, let T; be the set of all cells in T with values at most ft, and let 7) 
be obtained from T \ T, by transposing the shape (reflecting in an anti-diagonal) and 
replacing each entry i by N—i+1, where N = |6,| = ieee Clearly 7, and 7T> are 
shifted SYT. 
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Here is an example with n = 4 andt =5. 


1]2]4]6 6 

3{5/8|_, | [2]4 8 » (ERE bos isle!) 
7\9 3/151 (719 3/51 17 
10 10 


1{2/4} [1/2/3]5 

3|5|' [4 j 
Notice that, treating strict partitions as sets, 64 = (4,3,2,1) is the disjoint union of 
sh(7,) = (3,2) and sh(72) = (4,1). This is not a coincidence. 


Claim 14.8.17 Treating strict partitions as sets, 6, is the disjoint union of the shape 
of T, and the shape of To. 


In order to prove the claim notice that the borderline between 7; and T \ T) is 
a lattice path of length exactly n, starting at the northeastern corner of the staircase 
shape 6,, and using only S (south) and W (west) steps, and ending at the southwestern 
boundary. If the first step is S then the first part of sh(7;) isn, and the rest corresponds 
to a lattice path in 6,_1. Similarly, if the first step is W then the first part of sh(7>) is 
n, and the rest corresponds to a lattice path in 6,1. Thus exactly one of the shapes 
of 7; and T) has a part equal to n. The claim follows by induction on n. 

We deduce that, for any nonnegative integers n and t witht < (rs 


ye Se (14.1) 

ACEn 

|A|=t 
Here summation is over all strict partitions A with the prescribed restrictions, and A° 
is the complement of A in 6, = {1,...,n} (where strict partitions are treated as sets). 
In particular, the LHS is independent of f. 

Next apply the first bijection to SYT of truncated staircase shape 6, \ (1). Choose 
as a pivot the cell c = (2,n— 1), just southwest of the missing northeastern corner. 
The entry ¢ = T(c) satisfies 2n—3 <t < (5) —2n+2. Let T be a SYT of shape 6, \ (1) 
with T(c) =t. One subdivides the other entries of T into those that are (strictly) less 
than ¢ and those that are greater than t. The entries less than ¢ constitute T,. To obtain 
Th, replace each entry i > t of T by N —i+ 1, where N is the total number of entries 
in T, and suitably transpose the resulting array. It is easy to see that both 7, and 7) 
are shifted SYT. 


Example 14.8.18 


1|2|4 7 

3|5|7)_, | [2/4 618 » (LL12 14) [91817 
6/8 Bi. ° 3° 6 
9 9 
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Next notice that the shape of 7; is (n—1,n—3)UA while the shape of 7) is 
(n—1,n—3)UA‘, where A is a strict partition contained in 6,_4, A° is its comple- 
ment in 6,_4, and union denotes concatenation of partitions. 

We deduce that 


go\Q) = y » puro Eh olen ais . (14.2) 
t 2C8,—4 
|A|=t 
Here summation is over all strict partitions A with the prescribed restrictions. 
Finally, by Schur’s formula (Theorem 14.5.7), for any strict partitions 2, A° and 
MW = (W1,..-,Ux) with Wy >... > Ue > mand AUAS = by, the following holds. 


glk ght" = e(u,|A,|A°|) 948", (14.3) 
where 
ghm ot lSnl (Ml +1AL) (lm + 1/41)! 
ee (THI + Omn|) Ne|HA| Ae]! 
depends only on the sizes |A| and |A‘°| and not on the actual partitions A and 1°. 


Combining Equations (14.2), (14.1) and (14.3) together with some binomial iden- 
tities completes the proof. | 


c(u,|A|,A*]) = 


For a detailed proof and applications of the method to other truncated shapes 
see [1]. 

A different proof was presented by Panova [81]. Panova’s approach is sophisti- 
cated and involved and will just be outlined. The proof relies on a bijection from SYT 
of the given truncated shape to semi-standard Young tableaux of certain skew shapes. 
This translates the enumeration problem to the evaluation of sums of specializations 
of Schur functions. These evaluations are then reduced to computations of complex 
integrals, which are carried out by a comparison to another translation of the original 
enumerative problem to a volume of an associated order polytope. 


14.9 Rim hook and domino tableaux 
14.9.1 Definitions 


The following concept generalizes the notion of SYT. Recall from Section 14.3.3 the 
definition of a zigzag shape. 


Definition 14.9.1 Let r and n be positive integers and let 1 + rn. An r-rim hook 
tableau of shape A is a filling of the cells of the diagram |A] by the letters 1,...,n 
such that 
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1. each letter i fills exactly r cells, which form a zigzag shape called the i-th rim 
hook (or border strip); and 


2. for each 1 <k <n, the union of the i-th rim hooks for 1 <i<k is a diagram 
of ordinary shape. 


Denote by i the number of r-rim hook tableaux of shape A + rn. 


The nth rim hook forms a path-connected subset of the rim (southeastern bound- 
ary) of the diagram [A], and removing it leads inductively to a similar description for 
the other rim hooks. 

Note that 1-rim hook tableaux are ordinary SYT; 2-rim hook tableaux are also 
called domino tableaux. 


Example 14.9.2 Here is a domino tableau of shape (5,5,4): 


1/1/3|3|6 
2/4/5|5|6 
2/4/7|7 


and here is a 3-rim hook tableau of shape (5,4,3): 


1/1/3|3|3 
1/2/4|4 
2/214 


Definition 14.9.3 An r-partition of n is a sequence A = (A°,...,A'—!) of partitions 
of total size |A°| +... +|A’!| =n. The corresponding r-diagram [A°,...,A’~'] 
is the sequence ([A°],...,{A"~!]) of ordinary diagrams. It is sometimes drawn as a 
skew diagram, with [A‘] lying directly southwest of [A‘~"] for every 1 <i<r-l. 


Example 14.9.4 The 2-diagram of shape (A°,A') = ((3,1), (2,2)) is 


aah (FE) 


Definition 14.9.5 A standard Young r-tableau (r-SYT) T = (T°,...,7’~!) of 
shape A = (A°,...,A"~!) and total size n is obtained by inserting the integers 
1,2,...,n as entries into the cells of the diagram |A| such that the entries increase 
along rows and columns. 


14.9.2 The r-quotient and r-core 


Definition 14.9.6 Let A be a partition, and D = [A] the corresponding ordinary di- 
agram. The boundary sequence of A is the 0/1 sequence (A) constructed as fol- 
lows: Start at the southwestern corner of the diagram and proceed along the edges 
of the southeastern boundary up to the northeastern corner. Each horizontal (east- 
bound) step is encoded by 1, and each vertical (north-bound) step by 0. 
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Example 14.9.7 


L=CAy + Dea » 9(A) =(1,0,1,1,0) 


The boundary sequence starts with 1 and ends with 0, unless A is the empty 
partition, for which 0(/) is the empty sequence. 


Definition 14.9.8 The extended boundary sequence 0,(A) of A is the doubly- 
infinite sequence obtained from 0(A) by prepending to it the sequence (...,0,0) and 
appending to it the sequence (1,1,...). 


Geometrically, these additions represent a vertical ray and a horizontal ray, respec- 
tively, so that the tour of the boundary of [A] actually “starts” at the far south and 
“ends” at the far east. 


Example 14.9.9 /f A = (3,1) then (A) = (...,0,0,1,0,1,1,0,1,1,...), and if is 
empty then 0,(A) = (...,0,0,1,1,...). 


0. is clearly a bijection from the set of all partitions to the set of all doubly-infinite 
0/1 sequences with initially only 0-s and eventually only 1-s. 


Definition 14.9.10 There is a natural indexing of any (extended) boundary se- 
quence, as follows: The index k of an element is equal to the number of 1-s weakly to 
its left minus the number of O-s strictly to its right. 


Example 14.9.11 


0/1 sequence: 
Indexing: kee, OS 


Definition 14.9.12 Let A be a partition, r a positive integer, and s := 0,(A). 


1. The r-quotient g,(A) is a sequence of r partitions obtained as follows: For 
each 0 <i<r-—1 lets! be the subsequence of s corresponding to the in- 
dices that are congruent to i (mod r), and let A‘ := d,-1(s!). Then q,(A) := 
Ons Ae); 


2. The r-core (or r-residue) c,(A) is the partition A' = 0-'(s'), where s! is ob- 
tained from s by a sequence of moves that interchange a | in position i with a 
0 in position i+r (for some i), as long as such a move is still possible. 


Denote |q,(A)| := |A°|+...+|A7!]. 


Theorem 14.9.13 The equality |A| =r-|q,(A)|+|c,(A)| holds. 
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Example 14.9.14 For A = (6,4,2,2,2,1) andr =2, 
s =0,(A) =(...,0,0,0,1,0,1,0,0,0,1,1,0,1,1,0,1,1,1,...) 
with a hat over the entry indexed 0. It follows that 
s° =(...,0,0,0,0,0,1,1,0,1,...) and s'=(...,0,1,1,0,1,0,1,1,1,...). 
The 2-quotient is therefore qz(A) = ((2),(3,2)). The 2-core is 
c2(A) = dF !(s') = 071(...,0,0,0,0,0,0,0,1,0,1,0,1,1,1,1,1,1,1,...) = (2,1). 


Indeed, 


A| =17 =2-74+3 =2:-|qo(A)|+|c2(A)]. 
It is easy to see that, in this example, there are no 2-rim hook tableaux of shape A. 
Theorem 14.9.15 The following holds: f* 40 <=> the r-core c,(A) is empty. 
Example 14.9.16 Let A = (4,2), n =3 and r = 2. Then 
s=0,(A) =(...,0,0,0,1,1,0,1,1,0,1,1,1,...), 
so that the 2-core 
s' =01(...,0,0,0,0,0,1,1,1,1,1,1,1,...) 

is empty and the 2-quotient is 

go(A) = (07 1(...,0,0,1,1,0,1,...),a-1(...,0,1,0,1,1,1,...)) = ((2),(1)). 
Of course, here |A| =6 =2-3=r-|qo(A)|. 


In this example there are three domino tableaux of shape (4,2), and also three 
2-SYT of shape ((2), (1)): 


1} 1|2|2 1} 1|3 43 1 
3|3 * 12/2 ~ |] 


3 ae 2 | 


This is not a coincidence, as the following theorem shows. 


Theorem 14.9.17 Let A be a partition with empty r-core, and let q,(A) be its r- 
quotient. Then 


fi = par), 


Theorem 14.9.17 may be combined with the hook length formula for ordinary 
shapes (Theorem 14.5.3) to obtain the following. 
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Theorem 14.9.18 [53, p. 84] If f+ 40 then 


fre (JAl/r)! 

’ Teeta A):r|he he Jr 

Proof. Note that A has an empty r-core; let g,(A) = (A°,...,A’~!) be its r- 
quotient. By the hook length formula for ordinary shapes (Theorem 14.5.3) together 
with Observation 14.2.6, 


(A9,..,AT1) _ |A|/r yn ai (A |/r)! 
t =(iopenciart ny = Hi : 


c€[A9,...,arl] Mec 


A careful examination of the r-quotient shows that it induces a bijection from the 
cells in A with hook length divisible by r to all the cells in (A°,...,4’~!), such that 
every cell c € [A°,...,A’~!] with hook length h, corresponds to a cell c’ € [A] with 
hook length h, = rh,. This completes the proof. a 


Stanton and White [121] generalized the RS correspondence to a bijection from 
r-colored permutations (i.e., elements of the wreath product Z,?.%,) to pairs of r- 
rim hook tableaux of the same shape. The Stanton-White bijection, together with 
Theorem 14.9.17, implies the following generalization of Corollary 14.4.14. 


Theorem 14.9.19 The following hold: 


(1) (fhe arn 


Atrn 
and 
|n/2| 
(2) rm fi= py @ (Qe 1\ie*, 


In particular, the total number of domino tableaux of size 2n is equal to the num- 
ber of involutions in the hyperoctahedral group B,,. By similar arguments, the number 
of SYT of size n and unordered 2-partition shape is equal to the number of involu- 
tions in the Weyl group D,,. 

An important inequality for the number of rim hook tableaux has been found by 
Fomin and Lulov. 


Theorem 14.9.20 [33] For any A rn, 


1/r 
re 
freer "(5 : 


See also [68, 97, 65]. 
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14.10 g-Enumeration 


This section deals primarily with three classical combinatorial parameters—the in- 
version number, the descent number and the major index. These parameters were 
originally studied in the context of permutations (and, more generally, words). The 
major index, for example, was introduced by MacMahon [72]. These permutation 
statistics were studied extensively by Foata and Schiitzenberger [31, 32], Garsia and 
Gessel [39], and others. Only later were these concepts defined and studied for stan- 
dard Young tableaux. 


14.10.1 Permutation statistics 
We start with definitions of the main permutation statistics. 
Definition 14.10.1 The descent set of a permutation 1 € -Y, is 
Des(z) := {i: (i) > a(i+ 1)}, 
the descent number of 7 is 
des(z) := |Des(z)|, 


and the major index of 7 is 


The inversion set of 7 is 
Inv() := {(i,j): 1<i<j<n,m(i)>x(j)}, 
and the inversion number of 7 is 
inv(z) := |Inv(z)|. 


We also use standard g-notation: For a nonnegative integer n and nonnegative 
integers k,,...,k; with sum n, we write 


n 


a4 n n\q! 
tle 2 > tlt TT [es a = a 


q-1 i=l i=1 


Note that [0]! = 1. 


Theorem 14.10.2 (MacMahon’s fundamental equidistribution theorem [72]) For 
every positive integer n 


y gua) = y git) ~ [n]q! 


NESn TESp 


950 Handbook of Enumerative Combinatorics 


A bijective proof was given in the classical paper of Foata [31]. Refinements 
and generalizations were given by many. In particular, Foata’s bijection was applied 
to show that the major index and inversion number are equidistributed over inverse 
descent classes [32]. A different approach was suggested by Garsia and Gessel, who 
proved the following. 


Theorem 14.10.3 [39] For every subset S = {s1,...,8¢} C [n—1] (81 <... < Sg) 


y gral) = y i. = | n | ; 
q 


1eSn neSn 51,52 — $1,583 —S2,...,N—S, 
Des(a~!)cs Des(x~!)cs 


The following determinantal formula [119, Example 2.2.5] follows by the 
inclusion-exclusion principle. 


14.10.2 Statistics on tableaux 


We start with definitions of descent statistics for SYT. Let T be a standard Young 
tableau of shape D and size n = |D|. For each entry 1 <t <n let row(T~!(t)) denote 
the index of the row containing the cell T~!(t). 


Definition 14.10.4 The descent set of T is 
Des(T) := {1 <i<n—1|row(T'(i)) <row(T!(i+1))}, 
the descent number of T is 
des(T) := |Des(T)|, 
and the major index of T is 


maj(T) := y i. 


icDes(T) 
Example 14.10.5 Let 
1/2/5|8 
T=|3/4|6 
7 


Then Des(T) = {2,5,6}, des(T) = 3 and maj(T) = 2+5+6=13. 
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For a permutation 7 € .%,, recall from Section 14.4 the notation 7; for the skew 
(anti-diagonal) SYT that corresponds to 7 and the notation (P;,Q,) for the pair of 
ordinary SYT that corresponds to z under the Robinson-Schensted correspondence. 
By definition, 

Des(Tz) = Des(x~!). 


The jeu de taquin algorithm preserves the descent set of a SYT, and therefore 
Proposition 14.10.6 For every permutation 1 © -F,, 
Des(Pz) = Des(x~') and Des(Q,,) = Des(z). 


When it comes to inversion number, there is more than one possible definition 
for SYT. 


Definition 14.10.7 An inversion in T is a pair (i,j) such that 1 <i< j <nand the 
entry for j appears strictly south and strictly west of the entry for i: 


row(T!(i)) <row(T !(j)) and col(T~!(i)) > col(T~!(j)). 


The inversion set of T, Inv(T), consists of all the inversions in T, and the inversion 
number of T is 
inv(T) := |Inv(T)|. 


The sign of T is 
sign(T) := (—1)'™7), 


Definition 14.10.8 A weak inversion in T is a pair (i, j) such that 1 <i< j<nand 
the entry for j appears strictly south and weakly west of the entry for i: 


row(T~!(i)) <row(T !(j)) and col(T~!(i)) > col(T~!(j)). 


The weak inversion set of T, Winv(T), consists of all the weak inversions in T, and 
the weak inversion number of T is 


winv(T) := |Winv(T)|. 


Observation 14.10.9 For every standard Young tableaux T of ordinary shape i, 
7. 
inv(T) =inv(T Jy. 
winv(T ) = inv( +) (7) 


Here X’ is the length of the j-th column of the diagram [A\. 
Example 14.10.10 For T as in Example 14.10.5, we have 
Inv(T) = {(2,3), 2,0) (4,7); (3,7); (6,7)}, 


inv(T) =5 and sign(T) = —1. The weak inversion set consists of the inversion set 
plus all pairs of entries in the same column. Thus winv(T) = inv(T) + (3) + Gi + 


6) + (5) =44+34+14+1=9. 


For another (more complicated) inversion number on SYT see [47]. 
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14.10.3 Thin shapes 


We begin with refinements and g-analogues of results from Section 14.3. 


14.10.3.1 Hook shapes 


It is easy to verify that 


Observation 14.10.11 For any 1<k<n—-1 
xDes(T) = eh, 
sh(T)=(n—k,1*) 


where 
2 ry Xi, Xz 
<i) <in<...<ig<n—1 


are the elementary symmetric functions. 


Proof. Let T be a SYT of hook shape. Then for every | <i <n—1, iis a descent 


in T if and only if the letter i+ 1 lies in the first column. | 
Thus 
n—1 
y xbes(T) — TT (1+-x;). 
sh(T)=hook of size n i=1 
It follows that 
, n—1 ' 
res(7) gmail?) — TT (1+1q/). (14.4) 
sh(T)=hook of size n i=l 


Notice that for a T of hook shape and size n, sh(T) = (n —k, 1*) if and only if 
des(T) = k. Combining this observation with Equation (14.4), the g-binomial theo- 
rem implies that 


y gua = g@) ie x . 
sh(T)=(n—k,1*) q 


Finally, the statistics winv and maj are equal on all SYT of hook shape. For most 
Young tableaux of non-hook zigzag shapes these statistics are not equal. However, 
the equidistribution phenomenon may be generalized to all zigzags. This will be 
shown below. 


14.10.3.2 Zigzag shapes 


Recall from Section 14.7.1 that each subset S C [n — 1] defines a zigzag shape 
zigzag,,(S) of size n. The following statement refines Proposition 14.3.5. 


Proposition 14.10.12 For any S C [n— 1], 


fiiseen(S) =#{m © Y,: Des(m) =S}. 
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Proof. Standard Young tableaux of the zigzag shape encoded by S are in bijection 
with permutations in S, whose descent set is exactly S$. The bijection converts such a 
tableau into a permutation by reading the cell entries starting from the southwestern 
comer. For example, for S = {1,3,5,6}, 


216|8 
5 
‘eee ey, +> m= [914375268]. 


~ 


a 
Notice that in this example, 2~! = [274368591] and Des(7) = Des(z~!) = 
{2,3,6,8}. Also, Winv(T) = Inv(z~!). This is a general phenomenon. Indeed, 


Observation 14.10.13 Let T be a SYT of shape zigzag,,(S) and let 1 be its image 
under the bijection described in the proof of Proposition 14.10.12. Then 
S = Des(z), Des(T) = Des(z~'), Winv(T) = Inv(z'). 


By Observation 14.10.13, there is a maj-winv preserving bijection from SYT of 
a given zigzag shape to permutations in the corresponding descent class. Combining 
this with Theorem 14.10.3 one obtains 


Proposition 14.10.14 For a subset S = {51,...,s} © [n— 1] (51 < ... < sx), set 
SQ :=O and Sx41 :=n. Then 


; . 1 
maj(T) _ winv(T) _ [y I-det( 
y q y q [n|q [sj+1—Silq! 


sh(T)=zigzag, (S) sh(T)=zigzag, (S) 


k 


i,j=0 


14.10.3.3 Two-rowed shapes 


The major index and (weak) inversion number are not equidistributed over SYT of 
two-rowed shapes. However, the g-enumerations of both are nice. Two different g- 
Catalan numbers appear in the scene. First, consider enumeration by major index. 


Proposition 14.10.15 For everyn € N and0 <k <n/2 


ge es Ee heals 


sh(T )=(n—k,k) 
In particular, 
qr) = q"Cn(q) 
sh(T)=(m,m) 
where ‘ is 
m 
Cin eee ae 


is the m-th Fiirlinger-Hofbauer q-Catalan number [38]. 
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Corollary 14.10.16 The equality 


n 
Belial 
height(T)<2 \n/2]], 
holds. 


For a bijective proof and refinements see [10]. 
The descent set is invariant under jeu de taquin. Hence the proof of Theo- 
rem 14.4.17 may be lifted to a g-analogue. Here is a g-analogue of Theorem 14.4.17. 


Theorem 14.10.17 The major index generating function over SYT of size n and 
height < 3 is equal to 
[n/2) Fe 
mig)= Y a [ 5] Ca). 
k=0 q 


Furthermore, the following strengthened version of Corollary 14.4.14(3) holds. 


Corollary 14.10.18 For every positive integer k 
xDes(T) = xDes() 


9 


{TESYT,: height(T)<k} {me Avoid, (o;): 72=id} 
where Avoid,(0,) is the subset of all permutations in , that avoid the pattern 
oO; := [k,k—1,...,1]. 
Proof. Combine Theorem 14.4.13 with Proposition 14.10.6. a 


Counting by inversions is associated with another g-Catalan number. 


Definition 14.10.19 [19] Define the Carlitz-Riordan g-Catalan number C,,(q) by 


the recursion 
n 


Cri (q) = y gC (q)Cn-x(@) 
k=0 
with Co(q) := 1. 
These polynomials are, essentially, generating functions for the area under Dyck 
paths of order n. 


Proposition 14.10.20 [108] The equality 


ge!) & C (q). 
sh(T)=(m,m) 


holds. 
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Proposition 14.10.21 [108] For 0 < k <n/2 denote G;(q) := yx git), 
sh(T)=(n—k,k) 
Then sp) 
n/2 
n—2k ~ 
g! 2 )Gx(q?? = Gala): 
k=0 
Enumeration of two-rowed SYT by descent number was studied by Barahovski. 


Proposition 14.10.22 [8] Form>k > 1, 


14.10.4 The general case 
14.10.4.1 Counting by descents 


There is a nice formula, due to Gessel, for the number of SYT of a given shape A 
with a given descent set. Since it involves a scalar product of symmetric functions, 
which are out of the scope of the current survey, we refer the reader to the original 
paper [41, Theorem 7]. 

There is also arather complicated formula of Kreweras [62, 63] for the generating 
function of descent number on SYT of a given shape. However, the first moments of 
the distribution of this statistic may be calculated quite easily. 


Proposition 14.10.23 For every partitiond /}nand1<k<n-1 


Mi Mu 
1 Li(3)-Ei (3) ) oa 
#{T € SYT >k € Des(T)} = d 
(7 eSYT(A): ke Des(r)} = {5-2 ) 
Here i; is the length of the i-th row in the diagram [A], and Xj is the length of the 
j-th column. , 


For proofs see [50, 3]. 

One deduces that #{T € SYT(A) : k € Des(T)} is independent of k. This phe- 
nomenon may be generalized as follows: For any composition U = ({1,...,b,) of n 
let 

Su = {iM + H2,.-.,Hi+... + My-1} Cc [La- 1). 


The underlying partition of ju is obtained by reordering the parts in a weakly decreas- 
ing order. 


Theorem 14.10.24 For every partition A + n and any two compositions [ and Vv of 
n with the same underlying partition, 


= xDes(T)\Su x, xDes(T)\Sv _ 
T)=A T)=A 


sh( sh( 
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Proposition 14.10.23 implies that 


Corollary 14.10.25 The expected descent number of a random SYT of shape An 


is equal to i (E()-2)), 


The variance of descent number was computed in [3, 50], implying a concen- 
tration around the mean phenomenon. The proofs in [3] involve character theory, 
while those in [50] follow from a careful examination of the hook length bijection of 
Novelli, Pak and Stoyanovskii [76], described in Section 14.6.2 above. 


14.10.4.2 Counting by major index 


Counting SYT of general ordinary shape by descents is difficult. Surprisingly, it was 
discovered by Stanley that counting by major index leads to a natural and beautiful 
q-analogue of the ordinary hook length formula (Proposition 14.5.3). 


Theorem 14.10.26 (g-Hook Length Formula [114, Corollary 7.21.5]) For every par- 

titionAd tn ; 
: At 

qralT) = vi (7) [n]q! 


TESYT(A) TeejajlMelg 
This result follows from a more general identity, showing that the major index 


generating function for SYT of a skew shape is essentially the corresponding skew 
Schur function [114, Proposition 7.19.11]. If |A/u| =n then 


Lresyt(a/u) gn? 
os oo u 
saju(1, ag ed) = (1 =gl—@y-l =e) 


An elegant g-analogue of Schur’s shifted product formula (Proposition 14.5.7) 
was found by Stembridge. 


Theorem 14.10.27 [122, Corollary 5.2] For every strict partition A = (M1,...,A,) + 
n 
n-des(T)—maj(T) __ [n]q! ; qr —qhi 


4 ae gee ei 
TESYT(A*) i=1 [Ai] q! Grego 
Theorem 14.10.26 may be easily generalized to r-tableaux. 


Corollary 14.10.28 For every r-partition A = (A',...,A") of total size n 


: (Af nq! 
yg =e). et . 
TESYT(A) Teeayl clq 


The proof relies on a combination of Theorem 14.10.3 with the Stanton-White 
bijection for colored permutations. 
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14.10.4.3 Counting by inversions 


Unlike descent statistics, not much is known about enumeration by inversion statis- 
tics in the general case. The following result was conjectured by Stanley [116] and 
proved, independently, by Lam [64], Reifegerste [92] and Sjostrand [109]. 


Theorem 14.10.29 The equality 
y y sign(T) = 21"/2), 


AEnTESYT(A) 


holds. 


A generalization of Foata’s bijective proof of MacMahon’s fundamental equidis- 
tribution theorem (Theorem 14.10.2) to SYT of any given shape was described 
in [47], using a more involved concept of inversion number for SYT. 


14.11 Counting reduced words 


An interpretation of SYT as reduced words is presented in this section. This interpre- 
tation is based on Stanley’s seminal paper [112] and follow ups. For further reading 
see [15, §7.4-7.5] and [16, 87]. 


14.11.1 Coxeter generators and reduced words 


Recall that the symmetric group .%, is generated by the set S:= {s;: 1<i<n} 
subject to the defining Coxeter relations: 


s=1 (l<i<n);  sjsj=sjs ((i-i]>1)3  sisizisi =sipisisinn (1 <i<n—1). 


The elements of S are called simple reflections and may be identified with the adja- 
cent transposition in .%,, where s; = (i,i+1). 
The Coxeter length of a permutation 7 € .%, is 


C(x) :=min{t: s;,---s;, = 7}, 
the minimal length of an expression of 7 as a product of simple reflections. 
Claim 14.11.1 For every 1 € .Y%, 
£(z) =inv(z). 


A series of Coxeter generators (sj,,...,5;,) is a reduced word of 2 € .%, if the 
resulting product is a factorization of minimal length of 7, that is s;,---s;, = 7 and 
t = (z). In this section, the enumeration of reduced words will be reduced to enu- 
meration of SYT. 
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14.11.2 Ordinary and skew shapes 


A shuffle of the two sequences (1,2,...,k) and (k+1,k+2,...,@) is a permutation 
m € .Y; in which both sequences appear as subsequences. For example (k = 3, = 7): 
4516237. 


Proposition 14.11.2 [26] There exists a bijection A > n, from the set of all parti- 
tions to the set of all fixed point free shuffles such that: 


1. If A has height k, width (length of first row) €—k and size n then Ny, is a fixed 
point free shuffle of (1,2,...,k) and (k+1,k4+2,...,) with inv(m,) =n. 


2. The number of SYT of shape A is equal to the number of reduced words of T%,. 


Proof Sketch. _ For the first claim, read the permutation from the shape as follows: 
Encode the rows by 1,2,...,4 from bottom to top, and the columns by k+ 1,k+ 
2,...,4 from left to right. Then walk along the southeastern boundary from bottom 
to top. If the ith step is horizontal, set 7, (i) to be its column encoding; otherwise set 
7 (i) to be its row encoding. 


Example 14.11.3 The shape 


456758 


3 
2 
I 


corresponds to the shuffle permutation 
m = 41567283. 


For the second claim, read the reduced word from the SYT as follows: If the letter 
1 < j <n lies on the ith diagonal (from the left), set the jth letter in the word (from 
left to right) to be 5;. 


Example 14.11.4 The SYT 


45678 
3 1)/2}|3|6)8 
2 4/5)9|10 
1 


corresponds to the reduced word (in adjacent transpositions) 
$3545582838651875455 = 41567283. 


The proof that this map is a bijection from all SYT of shape / to all reduced 
words of 7, is obtained by induction on the size of 2. a 
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Corollary 14.11.5 For every pair of positive integers 1 < k < & the number of re- 
duced words of the permutation |k +1,k+2,...,€,1,2,...,k] is equal to the number 
of SYT of rectangular shape (k’~*). 


Proposition 14.11.6 There exists an injection from the set of all 321-avoiding per- 
mutations to the set of all skew shapes, which satisfies the following property: For 
every 321-avoiding permutation 1 there exists a skew shape A/[ such that the num- 
ber of reduced words of 1 is equal to the number of SYT of shape A /. 


The following theorem was conjectured and first proved by Stanley using 
symmetric functions [112]. A bijective proof was given later by Edelman and 
Greene [24]. 


Theorem 14.11.7 [112, Corollary 4.3] The number of reduced words (in adjacent 
transpositions) of the longest permutation wo := [n,n—1,..., 1] is equal to the number 
of SYT of staircase shape 6,—1 = (n—1,n—2,...,1). 


Corollary 14.11.5 and Theorem 14.11.7 are special instances of the following 
remarkable result. 


Theorem 14.11.8 [112, 24] 


I. For every permutation 1 € -%,, the number of reduced words of m can be ex- 


pressed as 
ym f* 
Atinv(z) 


where my are nonnegative integers canonically determined by 7. 


2. The above sum is a unique f» (i.e., my, = 64 ,,) if and only if m is 2143- 
avoiding. 


Reiner [93] applied Theorem 14.11.7 to show that the expected number of sub- 
words of types 5;5;4 15; and 5; 15;5;1 in arandom reduced word of the longest permu- 
tation is exactly one. He conjectured that the distribution of their number is Poisson. 
For some recent progress see [129]. 

A generalization of Theorem 14.11.7 to type B involves square shapes. The fol- 
lowing theorem was conjectured by Stanley and proved by Haiman. 


Theorem 14.11.9 [49] The number of reduced words (in the alphabet of Coxeter 
generators) of the longest signed permutation wo := [—1,—2,...,—n] in By is equal 
to the number of SYT of square shape (n"). 


For a recent application see [82]. 
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14.11.3 Shifted shapes 


An interpretation of the number of SYT of a shifted shape was given by Edelman. 
Recall the left weak order from Section 14.1.1, and recall that o covers 7 in this 
order if o = 5;a and £(o) = ¢(a) +1. Edelman considered a modification of this 
order in which we further require that the letter moved to the left be larger than all 
letters that precede it. 


Theorem 14.11.10 [23, Theorem 3.2] The number of maximal chains in the modified 
weak order is equal to the number of SYT of shifted staircase shape. 


A related interpretation of SYT of shifted shapes was given in [26]. A permuta- 
tion z € .Y, is unimodal if Des(z~!) = {1,..., j} for some 0 < j <n—1. Consider 
U,, the set of all unimodal permutations in .%,, as a poset under the left weak order 
induced from .%,. 


Proposition 14.11.11 [26] There exists a bijection A > my from the set of all shifted 
shapes contained in the shifted staircase 5,_; = (n—1,n—2,...,1) to the set U,, of 
all unimodal permutations in -Y, such that: 


1, |A| =inv(7). 


2. The number of SYT of shifted shape A is equal to the number of maximal chains 
in the interval lid, 7m] in Un. 


Proof Sketch. Construct the permutation 2, from the shape A as follows: Encode 
the rows of A by 1,2,... from top to bottom, and the columns by 2,3,... from left to 
right. Then walk along the southeastern boundary from bottom to top. If the ith step 
is horizontal, set 2, (i) to be its column encoding; otherwise set 7) (i) to be its row 
encoding. 


Example 14.11.12 The shifted shape 


23456 
1 
2 
3 
corresponds to the permutation 
m = 435621. 


Now construct the reduced word from the SYT T as follows: If the letter 7 lies in 
the ith diagonal (from left to right) of T then set the jth letter in the word (from left 
to right) to be s;. 
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Example 14.11.13 The SYT 


23456 
1 1/2/3|6|8 
2 4|5 | 9 |10 
3 7 


corresponds to the reduced word (in adjacent transpositions) 


$1828351S28481555354 = 435621. 


14.12 Appendix 1: Representation theoretic aspects 


Representation theory may be considered as the birthplace of SYT; in fact, one can- 
not imagine group representations without the presence of SYT. Representation the- 
ory has been intimately related to combinatorics since its early days. The pioneering 
work of Frobenius, Schur and Young made essential use of integer partitions and 
tableaux. In particular, formulas for restriction, induction and decomposition of rep- 
resentations, as well as many character formulas, involve SYT. On the other hand, it 
is well known that many enumerative problems may be solved using representations. 
In this survey we restricted the discussion to combinatorial approaches. It should 
be noted that most results have representation theoretic proofs, and in many cases 
the discovery of the enumerative results was motivated by representation theoretic 
problems. 

In this section we briefly point out several connections, assuming basic knowl- 
edge in non-commutative algebra, and give a very short sample of applications. 


14.12.1 Degrees and enumeration 


A SYTT of shape A has an associated group algebra element yr € C[.Y%,], called the 
Young symmetrizer. y7 has a key role: It is an idempotent, and its principal right 
ideal 

yrC[.A,| 


is an irreducible module of .%,. All irreducible modules are generated, up to isomor- 
phism, by Young symmetrizers, and two modules, which are generated by the Young 
symmetrizers of two SYT, are isomorphic if and only if these SYT have same shape. 
The irreducible characters of the symmetric group .~, over C are, thus, parameter- 
ized by the integer partitions of n. 


Proposition 14.12.1 The degree of the character indexed by At n is equal to f*, 
the number of SYT of the ordinary shape i. 
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This phenomenon extends to skew and shifted shapes. The number of SYT of 
skew shape A/L, fil # is equal to the degree of the decomposable module gener- 
ated by a Young symmetrizer of a SYT of shape A /. Projective representations are 
indexed by shifted shapes; the number of SYT of shifted shape A, eg". is equal to the 
degree of the associated projective representation. 

Most of the results in this survey have representation theoretic proofs or interpre- 
tations. A few examples will be given here. 


Proof sketch of Proposition 14.3.3. The symmetric group -%, acts naturally on 
subsets of size k. The associated character, pee), is multiplicity free; its decom- 
position into irreducibles is 


k 
rhe = Ly yr), (14.5) 
i=0 


L 


Hence 


n—kk) n—kk) _ ,,(n—k+1,k—1) 


er) = pl yu 


The degrees thus satisfy 
(n-k,k) 4, (n—kk) (7) (n-k (7) (nk kD yy _ (7) _ ff” 
fOHE = HI) = WOE) — a= (f)-(,",). 


This argumentation may be generalized to prove Theorem 14.5.4. First, notice 
that (14.5) is a special case of the Young rule for decomposing permutation mod- 
ules. The Young rule implies the determinantal Jacobi-Trudi formula for expressing 
an irreducible module as an alternating sum of permutation modules, see e.g. [53]. 
Evaluation of the characters at the identity permutation implies Theorem 14.5.4. 

Next proceed to identities that involve sums of f As, 


Proof of Corollary 14.4.14(1). Recall that for every finite group, the sum of squares 
of the degrees of the irreducibles is equal to the size of the group. This fact together 
with the interpretation of the fi-s as degrees of the irreducibles of the symmetric 
group -%, (Proposition 14.12.1) completes the proof. | 


The same proof yields Theorem 14.9.19(1). 

The Frobenius-Schur indicator theorem implies that for every finite group, which 
may be represented over R, the sum of degrees of the irreducibles is equal to the 
number of involutions in the group, implying Corollary 14.4.14(2). The proof of 
Theorem 14.9.19(2) is similar, see e.g. [18, 2]. 


Proof sketch of Corollary 14.4.16. |The permutation module defined by the action 
of 2, on the cosets of B, = Z2)-%, is isomorphic to a multiplicity free sum of all 
Soy-itreducible modules indexed by partitions with all parts even [69, §VII (2.4)]. 
Comparison of the dimensions completes the proof. | 
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14.12.2 Characters and g-enumeration 


The Murnaghan-Nakayama rule is a formula for computing values of irreducible 
-/,-characters as signed enumerations of rim hook tableaux. Here is an example of 
special interest. 


Proposition 14.12.2 For every A | rn, the value of the irreducible character x? at 
a conjugacy class of cycle type r” is equal to the number of r-rim hook tableaux; 
namely, 


Another interpretation of f- is as the degree of an irreducible module of the 
wreath product ZY). 

An equivalent formula for the irreducible character values is by weighted counts 
of all SYT of a given shape by their descents; g-enumeration then amounts to a 
computation of the corresponding Hecke algebra characters. 

These character formulas may be applied to counting SYT by descents. Here is a 
simple example. 


Proof sketch of Proposition 14.10.23. | By the Murnaghan-Nakayama rule, the value 
of the character * at a transposition s; = (i,i+ 1) is equal to 
l{T €SYT(A): i¢ Des(T)}|—|{T € SYT(A): ic Des(T)}| 
= f* —2\{T ESYT(A): ic Des(T)}|. 
Combining this with the explicit formula for this character [52] 
E(3)-, (7) 


Xr _ 
Maura) (a) 


fi 
completes the proof. | 


Finally, we quote two classical results, which apply enumeration by major index. 


Theorem 14.12.3 (Kraskiewicz-Weyman, in a widely circulated manuscript fi- 
nally published as [56]) Let @ be a primitive 1-dimensional character on the cyclic 
group Cy, of order n. Then, for any partition A of n, the multiplicity of y* in the 
induced character Inde @, which is also the character of the Sy, action on the mul- 
tilinear part of the free Lie algebra on n generators (and of many other actions 
on combinatorial objects) is equal to the number of T € SYT(A) with maj(T) = 1 
(mod n). 


This result may actually be deduced from the following one. 


Theorem 14.12.4 (Lusztig-Stanley) For any partition A of n and any0 <k < G), 


the multiplicity of x? in the character of the Sy, action on the k-th homogeneous 
component of the coinvariant algebra is equal to the number of T € SYT(A) with 
maj(T) =k. 


A parallel powerful language is that of symmetric functions. The interested reader 
is referred to the excellent textbooks [114, Ch. 7] and [69]. 
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14.13. Appendix 2: Asymptotics and probabilistic 
aspects 


An asymptotic formula is sometimes available when a simple explicit formula is 
not known. Sometimes, such formulas do lead to the discovery of surprising explicit 
formulas. A number of important asymptotic results will be given in this appendix. 

Recall the exact formulas (Corollary 14.3.4, Theorem 14.4.17 and Theo- 
rem 14.4.19) for the total number of SYT of ordinary shapes with small height. An 
asymptotic formula for the total number of SYT of bounded height was given by 
Regev [87]; see also [13, 117, 89]. 


Theorem 14.13.1 [87] Fix a positive integer k and a positive real number a. Then, 
asymptotically as n + , 


1 
Fa(n) vs y Ga ms f200 , y— 3 (k-1)(ak+2a-1) -c(k,Q), 
aye 


where 


k 
c(k, 00) := k2Mak+a ) (2a) a(k 1)(@k+1) (7) A(k-1)(2a oT] 


In particular, Titty 400 Fie, (2) !/ ” — &2 Important special cases are: & = 1, which 
gives (by Theorem 14.4.11 and Proposition 14.4.13) the asymptotics for the number 
of permutations in .Y, that do not contain a decreasing subsequence of length k + 1; 
and a = 1/2, which gives (by Corollary 14.4.14(3)) the asymptotics for the number 
of involutions in .~, with the same property. See [87] for many other applications. 

The proof of Theorem 14.13.1 uses the hook length formula for f is factoring out 
the dominant terms from the sum and interpreting what remains (in the limit n — 0) 
as a k-dimensional integral. An explicit evaluation of this integral, conjectured by 
Mehta and Dyson [75, 74], has been proved using Selberg’s integral formula [107]. 

Okounkov and Olshanski [77] introduced and studied a non-homogeneous ana- 
logue of Schur functions, the shifted Schur function. As a combinatorial applica- 
tion, they gave an explicit formula for the number of SYT of skew shape A /p. Stan- 
ley [115] proved a formula for f 4/H in terms of values of symmetric group characters. 
By applying the Vershik-Kerov .“%.-theory together with the Okounkov-Olshanski 
theory of shifted Schur functions, he used that formula to deduce the asymptotics of 
f*/#. See also [118] 

Asymptotic methods were applied to show that certain distinct ordinary shapes 
have the same multiset of hook lengths [90]. Bijective and other purely combinatorial 
proofs were given later [91, 14, 60, 44]. 

In two seminal papers, Logan and Shepp [67], and independently Vershik and 
Kerov [133], studied the problem of the limit shape of the pair of SYT that 
correspond, under the RS correspondence, to a permutation chosen uniformly at 
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random from .%,. In other words, choose each partition A of n with probability 
Un(A) = (f*)?/n!. This probability measure on the set of all partitions of n is called 
Plancherel measure. 

It was shown in [67, 133] that, under Plancherel measure, probability concen- 
trates near one asymptotic shape. See also [17]. 


Theorem 14.13.2 [67, 133] Draw a random ordinary diagram of size n in Russian 
notation (see Section 14.2.4) and scale it down by a factor of n'/2, Then, as n tends 
to infinity, the shape converges in probability, under Plancherel measure, to the fol- 
lowing limit shape: 


Oe 2(xarcsin§ + V4—x2), if |x| < 2; 
i, if | > 2. 


This deep result had significant impact on mathematics in recent decades [98]. 
A closely related problem is to find the shape that maximizes pe, First, notice 
that Corollary 14.4.14 implies that 


af ory Smarts: Aba} < val, 


where p(n) is the number of partitions of n. 


Theorem 14.13.3 [134] 


(1) There exist constants c, > cg > 0 such that 


eV nl < max{f* : AE n} < e OV" Val. 


(2) There exists constants c} = Co > 0 such that 


I nf 
OM aces CE oe he fee ri _ 
Bina Fant <a 1. 


Similar phenomena occur when Plancherel measure is replaced by other mea- 
sures. For the uniform measure see, e.g., [83, 84]. 

Motivated by the limit shape result, Pittel and Romik proved that there exists 
a limit shape to the two-dimensional surface defined by a uniform random SYT of 
rectangular shape [85]. 

Consider a fixed (i, j) € ZL and a SYT T chosen according to some probability 
distribution on the SYT of size n. A natural task is to estimate the probability that 
T (i,j) has a prescribed value. Regev was the first to give an asymptotic answer to 
this problem, for some probability measures, using .%.-theory [88]; see also [78]. A 
combinatorial approach was suggested later by McKay, Morse and Wilf [73]. Here 
is an interesting special case. 
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Proposition 14.13.4 [88, 73] For a random SYT T of order n and a positive integer 
k>1 


k-1 
Prob(T (2,1) =k) ~ [+ O(n-3/?). 
In [73], Proposition 14.13.4 was deduced from the following theorem. 


Theorem 14.13.5 [73] Let ut k be a fixed partition and let T be a fixed SYT of shape 
Lu. Let n > k and let N(n;T) denote the number of SYT with n cells that contain T. 
Then 
inf 

kl? 


where t, denotes the number of involutions in the symmetric group -Yy. 


N(n;T) ~ 


It follows that ; 
ia 
k) 
Abn 
Stanley [115], applying techniques of symmetric functions, deduced precise formulas 
for N(n;T) in the form of finite linear combinations of the fs. 
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15.1 Introduction 


The computer is an indispensable tool for a modern mathematician. It is not only 
useful for checking emails, or for writing articles, or for retrieving information from 
web services like Wikipedia, MathWorld, or Sloane’s celebrated online encyclope- 
dia of integer sequences (OEIS [170]). The computer is also useful for computing. 
People working in applied mathematics seem to be more aware of this than people 
working in pure mathematics, perhaps because of the immediate impact numerical 
computations have in natural sciences and engineering. While numerical computa- 
tions typically do not lead to rigorous proofs, symbolic computations do have proof 
quality. Symbolic computation and computer algebra can therefore be a valuable tool 
for people working in pure mathematics. And more than many other areas of pure 
mathematics, combinatorics can benefit from the advances that have been made in 
computer algebra in the past few decades. One reason may be that computer algebra 
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algorithms, just like all other algorithms, are applicable only to finite objects, and 
combinatorialists, unlike most other mathematicians, primarily study finite objects. 
Another reason may be that computer algebra algorithms developed during the past 
few decades are especially good at handling precisely those kinds of expressions, 
which tend to arise in the context of combinatorial problems. 

The purpose of this chapter is to sustain this claim. In Section 15.2 we give a short 
general overview of the fundamental concepts of computer algebra, with some facts 
that are of independent interest as well as some background that is used in the sub- 
sequent sections. Section 15.3 is devoted to techniques for calculating information 
about generating functions. We give only a very limited selection of methods that 
have been invented for certain special combinatorial problems and focus on more 
general techniques. Symbolic summation is discussed in Section 15.4. With combi- 
natorial problems leading invariably to all sorts of summation problems, the impor- 
tance of computer algebra algorithms for simplifying sums is obvious. Many sums 
arising in combinatorics can nowadays be routinely tackled by a computer algebra 
system, even some that are well beyond the scope of any reasonable hand calculation. 

The growing power of computer algebra in general, and especially its value for 
combinatorics, should not be underestimated. However, it should not be overesti- 
mated either. It is not realistic to expect that a serious research problem could some- 
how be solved autonomously by the computer after typing a few commands into 
a computer algebra system. Instead, solving a problem is most often an iterative 
process in which computer and user have to “collaborate.” This sometimes requires 
bringing the problem at hand into a form that is better suited for some computa- 
tional means, but possibly less useful for traditional paper-and-pencil reasoning. It 
sometimes also requires that several different computational techniques be applied 
together. One general and successful paradigm for solving problems by computer 
algebra is the guess-and-prove paradigm outlined in Section 15.5. 


15.2. Computer algebra essentials 


The characteristic feature of computer algebra algorithms is that they operate on 
exact representations of mathematical objects, as opposed to numerical algorithms, 
which accept approximate input and only produce approximate output. The advan- 
tage of working with exact representations is that the calculations are rigorous and 
the results have proof quality. The disadvantage is that exact computations may take 
much more time than approximate ones, and that some mathematical objects do not 
even admit a finite representation. 

In the present section, we give an overview over the most important domains in 
which exact computations are possible, and introduce the most important algorithms 
available for solving problems in them. Obviously, this section cannot be a thorough 
introduction to computer algebra, only a collection of some basic facts that are essen- 
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tial for the later parts of the chapter. Further information can be found in the classical 
textbooks on the subject [134, 78, 196, 184]. 


15.2.1 Numbers 
15.2.1.1_ Integers and rational numbers 


Perhaps the most natural domain in which exact computations can be done is the 
ring of integers. Obviously, every integer can be uniquely represented by a finite 
string of digits. This representation can be used for doing calculations (addition, 
multiplication, etc.) in the usual way. Rational numbers can be represented as pairs 
of integers (numerator and denominator), and we can do exact computations with 
those numbers as well. 

Integers may have many digits, and may hence consume a lot of memory on a 
computer. Longer integers also need more computation time. The most common op- 
erations on integers can be performed in quasi-linear time. This means there is some 
fixed integer k so that the operation can be performed on input with n digits in time 
O(nlog(n)*). In particular, addition, multiplication, division with remainder, greatest 
common divisor, least common multiple, and radix conversion can be performed in 
quasi-linear time [184, 42]. Factorization of integers is much harder. At the time of 
writing (2014), the fastest known algorithm requires exp(O(n!/3 log(n)7/*)) time to 
factor an integer with n digits. In practice this means that integers with fifty thousand 
decimal digits cannot be factored in any reasonable amount of time, but all the other 
operations mentioned before need virtually no time at all. 


15.2.1.2 Algebraic numbers 


An object & is called algebraic over a field F if it is the root of some univariate 
polynomial with coefficients in F. If F = Q we call & an algebraic number. For 
example, V2 is an algebraic number, as it is a root of x? — 2. 

An algebraic number can be represented exactly by a polynomial p of which it 
is a root (see Section 15.2.2 for more about polynomials) plus an approximate value 
that is sufficiently accurate to distinguish it from the other roots of p. For example, 
a possible representation of J2 would be [x? — 2,1] because 1 is closer to J2 than 
to the second root —/2. Root isolation algorithms [134, 151, 178, e.g.] may be used 
to determine the accuracy needed to distinguish the roots of a polynomial from each 
other, and to increase the accuracy of a given approximation if necessary. 

The set Q of all algebraic numbers forms a field, the algebraic closure of Q. 
To compute the sum or the product of two elements a, B € Q means that we know 
defining polynomials and approximate values for both of these values, and we want 
to know a defining polynomial and an approximate value for ~@+ B or @B. Adding or 
multiplying the two approximate values is of course easy. Defining polynomials can 
be constructed using Grébner bases or resultants. The use of Grobner bases for this 
purpose is explained in Example 15.2.7 below. For information on using resultants, 
see [36]. The construction of a defining polynomial for the multiplicative inverse is 
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easy: If po + pix +--+: + pax4 is the defining polynomial for a, then pg + pg_1x+ 
-+» + p1x4-! + pox4 is a defining polynomial for 1/a. 

The field K = Q(a) generated by a single algebraic number a is completely de- 
termined by the (unique) monic polynomial m of minimal degree of which @ is a 
root. If d is the degree of this polynomial, then K is a vector space of dimension d 
over Q, and a basis is given by 1,a@,a*,...,a@7~!. Consequently, the elements of K 
are precisely the elements p(@) where p is some polynomial of degree less than d. 
The (unique) polynomial p of degree less d is called the canonical representation 
of the element p(a) € K. If p,q are the canonical representations of two elements 
of K, then the canonical representation of their sum is given by p+q and the canon- 
ical representation of their product is given by rem(pg,m), where m is the defining 
polynomial of K. For computing the canonical representation of 1/p(q), use the ex- 
tended Euclidean algorithm for polynomials (see again Section 15.2.2) to find two 
polynomials s,t with ps + mt = 1. Then rem(s,m) is a canonical representation for 
1/p(a). 

There are algorithms for computing the Galois group of a given field Q(a@), and 
hence for analyzing the subfield structure of a given field. In particular, it is possible 
to decide algorithmically whether a given algebraic number can be expressed in terms 
of radicals. Details can be found in [64]. 


15.2.1.3 Real and complex numbers 


There is no way to do exact computations in R or C. For exact computations to be 
possible in some domain, it is necessary that there is a way of representing each 
of its elements in a finite way, i.e., as words of finite length over an alphabet with 
finitely many symbols. As there are only countably many words but uncountably 
many real or complex numbers, it follows that there is no way to exactly represent 
real or complex numbers, and hence there is also no way to do exact computations 
with them. 

When using approximate representations, the question is how accurate the output 
is. The design of algorithms for doing numerical computations aims at minimizing 
both the error (i.e. the distance between output and actual solution) as well as the 
runtime. Algorithms in this area typically rely on some iteration that produces a se- 
quence of approximate solutions that converges in the limit to the actual solution. 
The iteration is continued until the result is within the desired tolerance. The compu- 
tational cost is determined by the convergence rate and the cost of a single iteration. 

The design and analysis of such algorithms does not belong to the area of com- 
puter algebra but constitutes the area of numerical analysis. The two areas intersect 
at algorithms that take as input exact data and a target accuracy N € N and return 
output that is guaranteed to lie in a ball of radius 1/N around the true solution. Such 
techniques are known as arbitrary precision arithmetic, and they are used for com- 
puting mathematical constants such as 7 to billions of digits [29, 25, 42]. Note that 
with arbitrary precision arithmetic, it is always possible to recognize in a finite num- 
ber of steps that two numbers are distinct, but it is impossible to recognize that they 
are equal. 


980 Handbook of Enumerative Combinatorics 


Arbitrary precision arithmetic is relevant in combinatorial applications mainly in 
connection with asymptotic series. 


15.2.1.4 Finite fields and modular arithmetic 


Numbers with more than a few thousand digits will seldom arise directly in combi- 
natorial applications. However, it must be taken into account that numbers arising in 
intermediate results of a calculation can be much longer than those in the input or the 
output. This effect is known as expression swell. 

There is no expression swell in finite fields. If m is a positive integer, then the set 
mZ = {...,—-2m,—m,0,m,2m,...} C Zis an ideal in the ring Z, and so Z,, := Z/mZ 
has a natural ring structure. The elements of this ring are equivalence classes of 
integers subject to the equivalence relation a ~ b if and only if m | a—b. A set of 
representatives is given by {0,...,m—1}. If p is a prime, then Z, is a field. To find 
the multiplicative inverse of a nonzero element of Z,, represented by an integer a, use 
the extended Euclidean algorithm for integers to compute s,t € Z with as+ pt = 1. 
Then the equivalence class of s is the multiplicative inverse of the equivalence class 
of a. 

Finite fields are used to avoid expression swell. Instead of solving the problem 
in Z, we solve it in some finite field Z,. We then know the equivalence class mod- 
ulo p to which the answer belongs. If we know furthermore that the integer answer 
is bounded by M, then choosing a prime p > 2M will assure that every equivalence 
class contains only one element with absolute value at most M. This element must 
then be the solution to the integer problem. Applications of this technique are abun- 
dant. We illustrate it here only with some toy examples from linear algebra. Further 
applications can be found in the literature on computer algebra. 


Example 15.2.1 (Computing in modular images) Consider the linear system Ax = 


b where 
1570 —440 8048 —1328 
A={1775 1414 6723], b= | —1037 
1085 3436 621 283 


Suppose that for some reason we know that the system has a solution x = (x1 ,X2,x3) € 
Z> whose components are bounded by M = 10 in absolute value. Choose p = 23 > 
20 = 2M, interpret the entries of A and b as elements of Zp, and solve the linear 
system there. Then the coordinates of the solution vector also live in Zp. Select for 
each coordinate the unique representative whose absolute value is at most M. This 
gives the vector x = (4,—1,—1) € Z, which indeed satisfies Ax = b in Z. 


In general the solution of a linear system involves rational numbers. Also in this 
case, expression swell can be avoided by doing the computation modulo a prime. A 
rational number a/b with |a|,|b| <M can be reconstructed from its modular image 
ab~' mod p in Zp provided that p > 2M?. This is known as rational reconstruc- 
tion. The classical algorithm for rational reconstruction is based on the Euclidean 
algorithm and explained in Section 5.10 of [184]. An alternative approach based on 
lattice reduction is given in Example 15.2.5 below. 
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Example 15.2.2 (Rational Reconstruction) Consider the linear system Ax =c with 
A as in Example 15.2.1 and c = (296,817,1051) € Z?. Suppose that for some rea- 
son we know that the system has a solution x = (x1,X2,x3) € Q whose components 
have numerators and denominators that are bounded by M = 10 in absolute value. 
Choose p = 211 > 200 = 2M”, interpret the entries of A and c as elements of Zp, and 
solve the linear system there. The solution vector, expressed in terms of the canonical 
representatives, is (69,71, 141). Applying rational reconstruction to each coordinate 
gives x= (4, 3, 3) € Q’, which indeed satisfies Ax =c in Q. 


If we have the modular image of a number x with respect to several (coprime) 
moduli ,...,7, there is a way to merge them into a single modular image with 
respect to their product m mz ---me. This is called Chinese remaindering (see Sec- 
tion 5.4 of [184] or Section 5.6 of [78] for how this works). The use of several small 
moduli instead of one large modulus has several advantages. First, for hardware rea- 
sons arithmetic with integers smaller than 2% is significantly faster than arithmetic 
with larger integers. Second, the computation for different primes can be easily run 
in parallel on different machines. Third, in situations where an a priori bound of the 
solution is not available, we can incrementally increase the modulus by including 
more and more small primes. 


Example 15.2.3 (Chinese Remaindering) Consider the linear system Ax = d with 
A as in Example 15.2.1 and d = (296,817,1050) € Z°. We seek the solution 
x = (x1,x2,x3) € Q?. First choose p = 2147483647. Modulo this prime, the so- 
lution of the linear system, expressed in terms of canonical representatives, is 
(1146158679, 266089807, 564241799). Rational reconstruction applied to these 
numbers yields ( a, 506? RR ), which is not a solution of the system. This 
indicates that the modulus was chosen too small. Next choose q = 2147483629. Mod- 
ulo this prime, the solution reads (1645758289, 1234352967, 769474034). Using the 
Chinese remainder algorithm, we find that the solution modulo pq can be expressed 
as 


(572014109117159101, 1140337400488501 161, 793099489 171625462). 


Rational reconstruction applied to this modular image yields 


x= ( 295758998 58807909 Hees) 
= 219130875 ’ 87652350’ 43826175 /? 


which indeed satisfies Ax = d in Q. 


15.2.1.5 Lattice reduction 


Lattice reduction was originally invented as a subprocedure for the first polynomial 
time algorithm for factoring univariate polynomials with rational coefficients [120, 
184]. In combinatorics, it is mainly interesting for investigating possible relations 
among real numbers. 
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A lattice L is defined as a subset of Z” that is closed under addition and under 
multiplication by integers. A set {b,...,bq} C Z” is called a basis for L if 


L=Zb\4+---+Zbqg = {x1b) +--+ +xaba i x1,...,Xa € Z}. 


A lattice reduction algorithm takes as input some basis of a lattice L, and produces 
as output another basis for L with the feature that its elements are short with respect 
to the Euclid norm. For example, given the basis { (123,234), (345,456)} the output 
may be {(99, —12), (24,246)}. 

The computation of the shortest vector in a lattice is a computationally hard prob- 
lem that is not believed to be solvable by an efficient algorithm. However, there are 
algorithms that can efficiently construct a basis whose elements are perhaps not as 
short as can be but are at least guaranteed to be not much longer than the shortest 
ones, in a sense that can be made precise. The classical lattice reduction algorithm 
is known as LLL (named after the three inventors Lenstra, Lenstra, and Lovasz). A 
comprehensive account on the application of this algorithm as well as the underly- 
ing algorithmic details can be found in the monograph [137]. Here we only give two 
example applications that may be relevant in combinatorial applications. 


Example 15.2.4 (Integer relations) Let x,,...,x, be some real numbers. An inte- 
ger relation for x1,...,X, is a vector (€1,...,@n) € Z" such that ex, +--+ +enXp =0. 
For example, (33, —32, 16) is an integer relation for the three real numbers 


a con Bo 4, gn e 
a x2= i ’ x=) hae . 


n=1 n=1 


Integer relations can be discovered using lattice reduction. In the example, suppose 
we are able to compute the approximations 


x1 & E; := 1.01734306198445, 
X & Eo := 1.44494079843363, 
x3 & € := 0.791611531524342. 


Choose a big integer w, roughly of the reciprocal order of the approximation accu- 
racy, for instance w = 10> in our example. Consider the lattice L generated by 


{ (1,0,0, |w&i]), (0,1,0,|w&2]), (0,0,1, |w&3]) }. 
A lattice reduction algorithm may turn this basis into 


{ (33, —32, 16,1), (1898474, 452503, —3265793 4083522), 
(—2042705, —310957, 3192786, 6374529) }. 
The fact that the first vector in this basis is considerably shorter than the others is a 


strong indication (but of course not a proof) that its first components form an inte- 
ger relation for x1 ,x2,x3. It can be shown that every integer relation must eventually 
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appear in a reduced lattice basis if the procedure is repeated with higher and higher 
accuracy. Also, if there is no relation, lattice reduction can be used to prove state- 
ments of the form “if an integer relation exists, at least one of its coefficients e; must 
exceed ....” Results of this kind and further applications can be found in [30, 28, 15]. 
An alternative, more efficient but less general algorithm for finding integer relations 


is PSLQ [72, 73]. 


Example 15.2.5 (Rational and Algebraic Reconstruction) Suppose x € Z is the 
representative of an equivalence class modulo m € Z and we want to find u,v € Z 
such that x and * have the same image in Zi. To solve this problem via lattice re- 
duction, compute a short vector in the lattice generated by { (x,1), (m,0) }. Ifu 
and v are small compared to m, it is likely that (u,v) will show up in a reduced ba- 
sis. For example, for x = 9876540 and m = 12345678, applying lattice reduction to 
{ (9876540, 1), (12345678,0) } we may obtain { (—12,5), (365262,876614) }. We 
can take (u,v) = (—12,5). 

More generally, we can also use lattice reduction to recover an algebraic number 
from its modular image. Let be an algebraic number with a minimal polynomial 
of degree d, and let x € Q(a), say x = “te ia ioe for uo,..-,Ug-1,V € Z. 
Choose a large modulus m. An element & of Zyy is a suitable image of 0 if it satisfies 
the minimal polynomial of & modulo m. Such an element may or may not exist. If it 


exists, then we can also find an image xX of x in Zp. To recover x from X, compute a 
short vector in the lattice generated by 


1A Ones 
(m,0 
Here, by X/! we mean an arbitrary integer representative of the respective element 


Of Zm. If m is sufficiently large compared to v and the u;, a reduced basis of this 
lattice will contain the vector (ug,...,Ud—1,V): 


£30, 1), (0,8/ G,0y022j0,1), 4:25 (0):.150,4/07 1), 
.,0,0),(0,m,0,...,0,0),...... SO,2c0 Oar’. 


15.2.2 Polynomials 


For the remainder of this chapter, let R be a computable ring and K be a computable 
field. To be computable means that the elements admit a finite representation and al- 
gorithms are available for performing arithmetic operations and for deciding whether 
two elements are equal. 

By R[x1,...,Xm| we denote the ring of polynomials in the variables x,,..., x; with 
coefficients in R. This is a computable ring if R is. As a polynomial by definition 
has only finitely many terms, we can use a list of its monomials (coefficients and 
exponents) as a data structure. 

By K(x1,...,%m) we denote the fraction field of K[x),...,xm]. Its elements are 
called rational functions in x1,...,x;y. 
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15.2.2.1 Arithmetic and factorization 


Polynomials with coefficients in a ring can be added, multiplied, composed, differ- 
entiated, and compared to zero. When polynomials are represented as lists of terms, 
all these operations require O(n” log(n’”) log(log(n’”))) operations in the coefficient 
domain, where n is the degree of the output and m the number of variables. 

Univariate polynomials with coefficients in a field form a Euclidean domain: For 
every a,b € K|x], b £0 there exist (and we can compute) g,r € K|x] witha =qb+r 
and r = 0 or deg(r) < deg(b). We call quo(a,b) := q the quotient and rem(a,b) := 
the remainder of a and b. The number of field operations to perform division with 
remainder is roughly proportional to deg(a). 

In a Euclidean domain, we can compute the greatest common divisor g of two 
elements a and b by the Euclidean algorithm. With the extended Euclidean algorithm 
we can furthermore compute s,¢ such that g = as + tb. Again, the number of required 
field operations is roughly proportional to the size of the input. 

Factorization algorithms for Kx] are available for various fields, in particular for 
K =Z, (p prime), for K = Q and for K = Q(a@) (a algebraic over Q). Factoriza- 
tion of polynomials over any of these fields is considerably faster than factorization 
of integers. Observe that the factorization of a polynomial in general is different, 
depending on the assumed ground field. 


Example 15.2.6 (Polynomial Factorization) For p = x* — x? —2 € K|x] we have 
the following factorizations: 


(+1) @? =2) if K =Q 
(x +9)(x+17)(x+24) (x +32) if K = Za 
Get? ifK = Zz 
PY) (x—}(a3+a))(x+ b(a3+a@)) 
x(x— ¢(0°—5a))(x+ $(a3—Sa)) if K =Q(a) with a =i+ V2 
(7 -+1)@—1414...)(¢+1.414...) ifK=R 


Multivariate polynomials over a field do not form a Euclidean ring. There is nev- 
ertheless a notion of greatest common divisor, and an algorithm for computing it, but 
in general it is not possible to write the greatest common divisor g of two multivariate 
polynomials a and b as g = sa+tb for some other multivariate polynomials s and f. 

In some situations it is possible to reduce a computation for multivariate poly- 
nomials to one for univariate polynomials by selecting one of the variables and re- 
placing all the others by suitable powers of it. For example, consider a,b € K[x,y] 
and suppose that deg, (a) = deg,(b) = deg, (a) = deg,(b) =n. Then we can find the 
greatest common divisor of a and b by replacing y by x"*! in the input, obtaining uni- 
variate polynomials of degree n(n + 1), computing their greatest common divisor by 
the Euclidean algorithm, and then replacing each power x* by x"m™(&r+1) quolk.n+1) 
in the result. This technique is known as Kronecker substitution. 
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15.2.2.2. Evaluation and interpolation 


Polynomials can be evaluated at elements of the coefficient domain, the result being 
an element of the coefficient domain. Evaluating a given univariate polynomial of 
degree n at a given element of the ground ring costs O(n) arithmetic operations in 
the ground ring (using Horner’s rule). There is also an algorithm that evaluates the 
polynomial simultaneously at 1 given elements, using not much more than a linear 
number of operations in the ground ring [184, Chapter 10]. 

Conversely, given the values of a polynomial of degree n over a field at n+ 1 
distinct elements, we can find the coefficients of the polynomial by interpolation. 
This can be done using a number of operations that is quasi-linear in n. 

Evaluation and interpolation are used to avoid expression swell in K[x], very 
much like modular images and Chinese remaindering are used to avoid expression 
swell in Z. The value of a polynomial p € K [x] at an element e € K is the image of p 
under the map p+> p mod (x—e). Computing the interpolating polynomial for some 
given values (v1,v2,...,V,) at some given elements (e,¢2,...,@,) means to merge 
several images v; valid modulo linear polynomials x — e; into a joint image p valid 
modulo (x — e;)(x—e2)-+-(x— ey). 

Rational reconstruction is also available for polynomials. A rational function a/b 
with deg(a),deg(b) < M can be reconstructed from its image modulo p € K [x] pro- 
vided that deg(p) > 2M. 


15.2.2.3 Grébner bases 


If @ is a common root of two polynomials a and J, then it is also a root of a+b 
and a root of ca for every other polynomial c. The set of polynomials that have @ as 
a root is therefore an ideal of the polynomial ring K[x]. In the case of one variable, 
every ideal is generated by a single polynomial, the greatest common divisor of all 
the elements of the ideal. This is no longer true in the multivariate case. Here we 
only have Hilbert’s basis theorem, which says that every ideal J C K[x,...,xn] is 
generated by a finite number of elements, i.e., there exist b1,...,m € K[x1,...,Xn] 
such that p € J if and only if there exist pj,..., pm € K[x1,...,%n] such that p = 
pid) +--+ + pmbm. We write (b1,...,bm) :=1 for the ideal generated by b1,...,Dm- 

The basis of an ideal is not uniquely determined, and it is in general not easy to 
see whether two given bases generate the same ideal or not. A Grébner basis [49] is 
a distinguished ideal basis. In a sense, Grébner bases play the role among ideal bases 
that matrices in reduced echelon form play among matrices. Given an arbitrary basis 
of an ideal 7, a Grobner basis for J can be computed using Buchberger’s algorithm. 

Space limitations prevent us from giving a thorough introduction to Groébner 
bases at this point. The interested reader is referred to the classical textbooks on 
the subject [66, 21, 7] or to the introductory articles [50, 51]. We list here only some 
of the problems that can be solved computationally using Grobner bases. Given poly- 
nomials p,b1,...,bm,C1,---,C¢ © K[x1,...,%n], we can do the following: 


e Decide whether p € (bj,...,bm), and in the affirmative case, find cofactors 
(“certificates”) p1,...,Pm € K[x1,.-.,Xn] with p = pib) +--+ + pmb. 


986 Handbook of Enumerative Combinatorics 


e Decide whether there exists a positive integer k such that p* € (by,...,Dm), 
and in the affirmative case, find such a k. 

e Decide whether (b1,...,0m) = (c1,---,C¢)- 

e For any givenk € {1,...,2—1}, compute a basis for the ideal (b1,...,Bm)M 
K{x1,...,x,] in the smaller ring K[x,,...,x,]. 

e Compute a basis for the ideal (b1,...,bm) A (c1,...,€¢) C K[x1,.--,%n]. 

e Compute a basis of the ideal ker(@), where @: K[y1,...,¥m] > K[x1,.--,%n] 
is the ring homomorphism that leaves K fixed and sends each y; to bj. 

e Decide whether p € img(@) = K[by,..., by], and in the affirmative case, find 
a polynomial g € K|y1,...,¥m] such that p = q(b1,...,bm). 

e Compute a basis of the so-called syzygy-module consisting of all vectors 
(P1,---;Pm) © K[x1,..-,X,]” with the property pjb) +--+ + pmbm = 0. 

e Decide whether the number of points (0,...,Qn) € K" for which all the b; 
are simultaneously zero is finite, and if yes, list all these points. 

e Determine the Hilbert dimension of the ideal (b1,...,Dm). 


Example 15.2.7 (Algebraic number arithmetic) Let a, B € Q and let p,q € Q|x| 
be polynomials with p(a) = q(B) = 0. Suppose we want to find a polynomial r € Q{x] 
with r(y) = 0, where y = aB. This can be done using Grébner bases, as follows. 
Consider polynomials in Q|x, y,z| in which the variables x,y,z are meant to represent 
the algebraic numbers 0, B , y, respectively. For every polynomial u € (p(x),q(y),z— 
xy) C Qix, y,z] we have u(a,B,7) = 0, because this is true for the generators by 
definition, and the definition of ideals implies that it is also true for every other 
element. Use Grébner bases to compute a basis of the ideal (p(x), q(y),z—xy) NQ|z]. 
This basis will consist of a single polynomial r € Q|z] with the property r(y) = 0. 

More generally, if y = g(a, B) where g = “ € Q(x,y) is a rational function with 
v(a@, B) 40, then we can find an annihilating polynomial for y by computing a basis 
of the ideal (p(x), q(y),v(x,y)z — u(x, y)) AQ]. 


Example 15.2.8 (Equational Reasoning) Suppose we want to prove the formula 
Vx,y,zEC: (Gytycter= OA +y?+2°-4=0) 
=> (yr? tyz+2 —2y—2z=O0Vy? +yz+2? +2y4+2z= 0)). 


Translated into geometry, the claim is that the set of common roots in C? of the two 
polynomials in the first line is contained in the set of all points C? where at least one 
of the two polynomials in the second line vanishes. The latter set is identical to the set 
of all roots in C3 of the product p := (y? +yz+2 —2y —22)(y* +yz +2 +2y4+2z). 
By Hilbert’s Nullstellensatz [66, Section 4.1], the claim is true if and only if there 
exists a positive integer k with p* € (xy +-yz+zy,x7 +y? +2 —4) CC[x,y, 2]. It can 
be checked by a Grébner basis computation that k = 1 does the job. Grébner bases 
can also be used to compute the certificate relation 


p = (yz—xy — xz) (xy tyz+2x) + (y? +2yz +2) (0? +? +27 —4). 
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15.2.2.4 Cylindrical algebraic decomposition 


Groébner bases are useful for studying the solution sets of systems of polynomial 
equations. Such solution sets are called algebraic sets. In a field with an order re- 
lation <, we can also consider polynomial inequalities. The solution set of a sys- 
tem of polynomial equations and inequalities is called a semi-algebraic set. Prob- 
lems related to such sets can be solved using Cylindrical Algebraic Decomposition 
(CAD) [65, 53], also known as Collins’s algorithm. 

Again, space is too short to give a thorough introduction into this technique. We 
only mention the most relevant problems that can be solved by CAD and give two 
examples. Readers interested in using this technique are referred to the tutorial pa- 
per [98]. Algorithmic details can be found in the textbook [18]. If semi-algebraic sets 
are represented through a defining system of polynomial equations and inequalities, 
CAD can be used to: 


e Decide whether or not a given semi-algebraic set is empty, finite, open, 
closed, connected, or bounded. 


e Decide whether or not a given semi-algebraic set is contained in another one. 
e Determine the (topological) dimension of a given semi-algebraic set. 

e Determine a sample point of a given nonempty semi-algebraic set. 

e Determine the number of points of a given finite semi-algebraic set. 

e Determine a tight bounding box of a given bounded semi-algebraic set. 

e Determine the connected components of a given semi-algebraic set. 

e Determine the boundary, closure, or interior of a given semi-algebraic set. 


e Determine the projection of a given semi-algebraic set in IR” to a coordinate 
subspace R* (k <n). 


e Determine the semi-algebraic set consisting of all points (11,...,%n-1) € 
R’-! such that for all x, € R the given semi-algebraic set contains 
(X1,---;Xn—1,Xn)- 


CAD supports quantifier elimination. This means that if @ is a formula built 
from polynomials in a finite number of variables x,,...,x, and y1,...,¥m, Comparison 
symbols =, >, <, >, <, logical connectives A, V, etc., and quantifiers V, 4 according 
to the usual syntactic rules, and if y;,...,y,, are the variables bound by the quantifiers 
and x1,...,%, are the free variables, then there exists (and CAD can compute) another 
formula ¥ of polynomial inequalities in x),...,x, only with the property that 


Vxq,---,%n ER: ®(xy,...,%)) <> VY(x1,.--, Xn). 


Example 15.2.9 (Quantifier Elimination) Applied to the quantified formula 


®:=Vae RIDER: (a—2)(b—-2) >c 3a? +b°-c? <0, 


CAD produces the equivalent formula ¥ :=c <—2Vc=0. 
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Example 15.2.10 (A Fibonacci Inequality) Sometimes it is possible to use CAD for 
proving inequalities about quantities that are not polynomials. As an example, con- 
sider the following inequality, where F,, denotes the nth Fibonacci number defined by 
Fo = 0, Fy = 1, Fn42 = Frnt + Fn (n 2 9): 


oy (2Fevt = Fie)? (Fn +1 + Fn = 3)? (n>2) 


2 Fy a Frat tin — 1 — 


We prove this inequality by induction on n. The validity for n = 2 is easily checked. 
Now assume the inequality holds for some n = 2. We have to show it also holds for 
n+1 in place of n, i.e., 


y a Fe Ss (3Fi42 + Fri — 3) 
k=1 _ Fi a+, il 


which, using ae; ag = Vp Ak + An41 and the Fibonacci recurrence, can be rewrit- 
ten to 


y (2Fiai— Fi? | Fay +2Fe)?? . Fay +3F.-3) 


fo Fr Fri 2Fn4itFn—1 


Now in order to complete the proof of the induction step, it suffices to prove the 
formula 


47 3)2 27/2 4 _ 2)2 

Vany.2€R: (x>0Ay>z>1Ar> PFE) 4 OF é) Sans 2) 
yergead y 2y+z—1 

(2Fk41 Fi)” 


in which the variables x,y,z are used as representations of the sum Yr_, Fi ; 
the number F,4, and the number F,, respectively. CAD has no trouble proving this 
formula (as an equivalent quantifier free formula it returns “true”). Further appli- 
cations in this direction are described in [80, 96]. 


15.2.3. Formal power series 


The ring R{[x]] of formal power series in x is defined as the ring of all infinite se- 
quences (an )”_ 0 equipped with termwise addition as sum and with the convolution 
(dn) po (On) 0 = (YF arn ile. _9 as multiplication. It is customary to use the no- 
tation ) 9 dx” instead of (Gi)" 5: 

The ring R[[x]] is not computable, even if R is, for the same reason for which R 
is not computable: Since any ring R has at least two elements, there are uncountably 
many sequences in R, viz. uncountably many formal power series, but there are only 
countably many words over a finite alphabet at our disposal for their representations. 
Computations in R[[x]] are therefore necessarily limited. 


15.2.3.1 Truncated power series 


The first possibility for computing with formal power series is to restrict the attention 
to a certain fixed number, N, of terms, and discard the remaining ones. A formal 
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power series may then be written as aj +a,x+--- +ay_xN7} +O(x%), where the 
symbol O(-) does not have any analytic meaning but only indicates the truncation 
order. A formal power series of which only the first N terms are known, for some 
finite N €N, is called a truncated power series. 

It follows straight from the definition that if we know the first N terms of two 
formal power series a and b, then we can calculate the first N terms of their sum 
and their product. From the first NV terms of a we can also compute the first N — 1 
terms of the (term-wise) derivative a’ of a and, if Q C R, the first VN + 1 terms of the 
(term-wise) integral fa of a. 

If a is a formal power series whose zeroeth term admits a multiplicative inverse 
in R, then a itself admits a multiplicative inverse in R[[x]], and we can compute its 
first N terms if we know the first NV terms of a. Furthermore, if b is a formal power 
series whose zeroeth term is 0, then the composition boa = a(b(x)) € R{[x]] is well- 
defined, and we can compute its first V terms if we know the first NV terms of both a 
and b. See, for example, [43, 42] for classical as well as more advanced algorithms. 

The arithmetic for truncated power series has some similarities with arithmetic 
for polynomials. Note however that for a truncated power series, the terms beyond 
the truncation order N are unknown, whereas for a polynomial of degree N the 
higher order terms are zero. Algebraically, truncated power series of order N can be 
identified with elements of the quotient ring R[x] /(x’). 


15.2.3.2 Lazy power series 


Truncated power series are in R[[x]] what fixed precision floating point numbers are 
in R. In the same sense, lazy power series arithmetic is in R[[x]] what arbitrary preci- 
sion arithmetic is in R. The general idea is that a formal power series is represented 
by an algorithm which can calculate the nth coefficient of the series, for any given n. 
This representation is called lazy because it is in principle capable of computing any 
coefficient of a series but does not do so until explicitly asked for it. 

It is clear that we can do arithmetic using the lazy representation. For example, 
when we know lazy representations for two series a and b, the sum a+b can be 
represented by an algorithm that computes the nth term of a+ b by first computing 
the nth term of a using the lazy representation of a, then computing the nth term of 
b using the lazy representation of b, and then returning their sum. Likewise for the 
other operations. 

Lazy representations allow for recognizing in a finite number of steps that a given 
formal power series is different from zero: we only need to enumerate all its coeffi- 
cients, one after the other, until we encounter a nonzero coefficient. This procedure 
will run forever if and only if the series at hand is the zero series. In contrast, there 
is no way to tell whether a formal power series represented by a truncated series of 
order N whose first N coefficients are zero is nonzero or not. 

A common design pattern for lazy representations is to store a functional equation 
for the series in question (e.g., a differential equation satisfied by the series, or a 
recurrence equation for the coefficient sequence), plus possibly some initial values 
that together with the equation uniquely determine the whole object and allow for 
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computing any desired number of additional terms. For example, the formal power 


series a:= V1 —x—x? = 1— Sx— 32° +--- € Q[[y]] is uniquely determined by the 


equation a? — (1 — x —x”) = 0 together with the initial value ap = 1, as follows for 
instance from the implicit function theorem for formal power series [171, 102]. 


15.2.3.3 Generalized series 


If R is an integral domain, then so is R[[x]]. However, R[[x]] is not a field, even when R 
is, because the element x does not have a multiplicative inverse in R[[x]]. For a field K, 
the quotient field of the ring K[[x]] is denoted K((x)). Its elements can be identified 
with formal series of the form a a,x" with no € Z some (possibly negative) in- 
teger. These series are called formal Laurent series. If a is a formal Laurent series, 
then [x°]a is defined as the formal power series obtained from a by discarding all 
(finitely many) terms with negative exponent. 

Puiseux series are series of the form y ae px"! " where no € Z and r is a positive 
integer, called the ramification index of the series. When K is algebraically closed, 
then the set of all Puiseux series, sometimes denoted by K((x)), is again algebraically 
closed [185]. Puiseux series arise as expansions of algebraic functions, i.e., roots of 
univariate polynomials over a rational function field. 

Other types of infinite series may arise as solutions of linear differential equations 
with polynomial coefficients. So-called Fuchsian equations [88] have series of the 
form x%a where & is some constant and a is a formal power series. More generally, 
solutions of differential equations admit generalized series solutions of the form ea, 
where e is a certain closed form expression and a is a formal power series. See the 
remarks on solutions in Section 15.2.4 for a precise description of the closed form 
part. 

Laurent series, Puiseux series, and some other types of generalized series have in 
common that they are determined by a formal power series plus some finite amount of 
additional data. Therefore, for their computational treatment similar remarks apply as 
for formal power series. Even more general series arise as asymptotic expansions of 
elementary functions or solutions of certain transcendental equations. For definitions 
and algorithmic techniques, see [153, 165, 179] and the references given there. 


15.2.3.4 The multivariate case 


The ring R[[x1,...,%n]] of formal power series in n variables is defined in the obvious 
way. Concerning computations, similar remarks apply as for univariate power series. 
In particular, a power series admits a multiplicative inverse in R[[x,,...,x,]] if and 
only if the coefficient of oR -x? admits a multiplicative inverse in R. Furthermore, 
a multivariate power series b € R[[x1,...,%n]] may be substituted for y; into a power 
series a € R[[y1,...,¥m]], giving a power series in R[[x1,...,Xn,y2,---;Ym]], provided 
that the coefficient of ae ---x9 in b is zero. 

As in the univariate case, there is in general no way to represent a formal power 
series in finite terms, but we can always do computations with truncated series, and 
under suitable circumstances there may be a way to define lazy series. Instead of a 
single truncation order, we may now have individual truncation orders for each vari- 


Computer Algebra 991 


able. Alternatively, one can select a fixed tuple (w1,...,w,) of positive real numbers, 
called a weight vector, and say that truncating a power series at order N means to 
list all the coefficients aj,__;,, with O < wij +---+Wnin < N. Note that for every 
fixed N, there are only finitely many. 

In order to allow also negative exponents, we can proceed as follows. Let 
C C R" be a pointed cone, i.e., a set with the properties 0 € C and for all x,y EC 
and all A,u > 0 we also have Ax + uy € C and x €CA-xE€C>x=0. It 
can be shown that the set Rc[[x1,...,xn]]| consisting of all formal infinite sums 
Le Cit jeein)ECNZ” ia ae ---x/n together with the natural addition and multiplica- 


tion forms a ring. 


Example 15.2.11 (Series Expansions of Multivariate Rational Functions) For ev- 
ery rational function r € Q(x1,...,Xn) it is possible to find a cone C and a vector 
(é1,..-,€n) € Z" so that r admits a series expansion in 


+ 
xil x82 ++ x"Qel[x1,...,Xn]] = { y Cie La e] ve xtnten iin oe a}. 


(V1 ,---Yn EC 


The cone C is usually not unique, and depending on the choice of C, the coefficients 


in the expansion may be different. For example, consider r = si € Q(x,y). Taking 


C= {(A,u) €R?:A >0,u <0} andC’ = {(A,p) € R?:A <0," > O}, we have 


the two expansions 


1 
—— ay baa try Pury -€ Qelbey] 
x+y 
ax ax yta ty x4 te € Qe[lx,y]]. 
For a series a in some ring Rc|[x1,-..,%n]], the positive part [xpxz ---xZ]a is 
defined as the formal power series obtained from a by discarding all the terms 
ey e een Ea 
cx, +++x;" where not all the exponents e),...,@, are positive. The positive part al- 
ways belongs to the usual multivariate polynomial ring R[[x1,...,%n]]. 


A detailed introduction to rings of multivariate formal series involving negative 
exponents is given in [13]. 


15.2.4 Operators 


Just as algebraic numbers are represented exactly in a computer by the polynomial 
whose root they are, together with an approximation that is accurate enough to distin- 
guish the number in question from the other roots of the polynomial, we can represent 
certain functions and infinite sequences by the differential equations or recurrence 
relations whose solutions they are, together with a suitable number of initial values. 
Linear operators that act on functions or sequences form rings which resemble the 
usual polynomial rings. 
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15.2.4.1 Ore algebras 


An Ore algebra is a ring of polynomials with a noncommutative multiplication that 
is defined in such a way that the product of two elements of the algebra applied to 
some function is the same as applying the two elements one after the other to this 
function. The general definition makes use of the notion of o-derivation. If R is a 
ring and o: R — R is aring homomorphism, then 6: R > R is called a o-derivation 
if we have 6(a +b) = 6(a) + 6(b) and 6(ab) = 6(a)b+ o(a)6(b) for all a,b € R. 
Ore [140] introduces the following definition. Let R be aring, let 0),...,0,: R- 
R be endomorphisms, and let 6;,...,6,: R — R be such that 6; is a o;-derivation 
(Gi = 1,...,n). The ring R[O),...,0,] of all (multivariate) polynomials in 01,...,0n 
with coefficients in R, together with the usual (commutative) addition and the unique 
multiplication satisfying the commutation rules 0,0; = 0;0; for all i,j and dja = 
0;(a)0; + 6;(a) for i= 1,...,n and alla € R is called an Ore algebra over R. 


Example 15.2.12 (Recurrence, g-Recurrence, and Differential Operators) —_/. 
The elements of the Ore algebra K(x)[0] with o: K(x) — K(x) defined via 
(x) =x+1 and o(c) =c for all c € K, and with 6 = 0 can be interpreted 
as recurrence operators. In this case, we usually prefer to use the symbol n 
instead of x and the symbol S,, instead of 0. 


2. Let q € K \ {0}. The elements of the Ore algebra K(x)[0] with o(x) = qx and 
o(c) =c for all c € K, and with 6 = 0 are called q-recurrence operators. 


3. The elements of the Ore algebra K(x)[0| with o being the identity on K(x) and 
6= £ can be interpreted as differential operators. In this case, we usually 
write D, instead of 0. 


Note that by repeated application of the commutation rules, we can write every 
element of an Ore algebra as a finite sum of monomials wd}! ---d¢" with u in the 
coefficient domain and ¢1,...,e, € N. 

If the coefficient domain is a field K and there is only one generator 0, as in the 
examples above, the Ore algebra is a left Euclidean domain. This means that for any 
given A, B € K[0] we can find two other operators Q,R € K[O] withA = OB+R, with 
R being zero or having smaller order than B. Also the greatest common right divisor 
gcrd(A,B) and the least common left multiple Iclm(A,B) of A,B can be computed. 
See [48, 5] for an introduction into the classical algorithms, and [89, 34] for some 
more recent developments. 

If the coefficient domain is not a field or if there is more than one generator, then 
the Ore algebra is in general no longer a left Euclidean domain, but there is still a 
notion of Grdbner bases for left ideals in such algebras, and there are algorithms 
for computing them. In particular, every left ideal in an Ore algebra over a ring R is 
generated by a finite basis (provided that this is the case for ideals in the coefficient 
domain R). For details, see [63, 110]. 
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15.2.4.2 Actions and solutions 


Elements of Ore algebras are used to represent functions that they map to zero. To 
make this precise, fix an Ore algebra A = R[01,...,0,]. Let M be some R-module, 
and for each 0; choose a function d;: M — M that satisfies certain compatibility 
conditions so that the action A x M > M, (a,f)» a-f defined by a- f =af for 
aé Rand f €M and O;: f =d,(f) fori=1,...,n and f € M satisfies the laws 
(a+b)-f =(a-f)+(b- f) and (ab). f =a-(b- f) for alla,b € A and f € M. 


Example 15.2.13 (Some Natural Actions) = 7. Consider the Ore algebra A = 
Zn][Sp], and let M = CN be the set of all infinite sequences (an)*_» of com- 
plex numbers. With the natural action of A on M, an operator L = f9(n) + 
€\(n)Sp+---+2£,(n)S), applied to a sequence (an) in M yields the sequence 
(€o(n)an +26) (n)ane tee + £-(n)antr),_o in M. 


2. For K = Q(q), let A = K(x)[0] be the Ore algebra from Example 15.2.12.(2). 
Let M = KN be the set of all infinite sequences (an) 9 of rational func- 
tions in q. The algebra A acts on M if we let x- (an)? 9 := (q"an)_9 and 
9 - (An)p=0 *= (4n+1)n=0- 


3. The Ore algebra A = Z|x]|D,]| acts in a natural way on the ring M = C|[x]] of 
formal power series with complex coefficients. It also acts in a natural way on 
the field M of meromorphic functions. 


4. The Ore algebra A = Q|x,n]|[Dx,Sn] acts in a natural way on the Q-vector 
space of all functions f: C x N > C that are analytic with respect to the first 
variable. 


An element f € M is called a solution of an operator a € A if a- f = 0. For a fixed 
operator a € A, we call V(a) :={ f € M|a- f =0} the solution space of a in M. If 
C CR denotes the ring of all elements that are invariant under all the 0; and mapped 
to zero by all the 6; that define the Ore algebra A, then V(a) is a C-submodule of M. 
For a fixed element f € M, we call ann(f) := {a €A|a-f =0} the annihilator of 
f inA. This is a left ideal of A. 

If A = K[0] for some field K, then we have V(gcrd(a,b)) = V(a) N V(b) and 
V(lclm(a,b)) C V(a) + V(b). The latter inclusion becomes an equality if M is suffi- 
ciently rich. 

There are various algorithms for finding “closed form” solutions of a given opera- 
tor a € K(x)[0], where K is a field of characteristic zero left fixed by o and mapped to 
0 by 6. In particular, we can find a basis of the K-vector space V (a) NK (x) of all ratio- 
nal functions f € K(x) with a- f =0[1, 2, 182, 45]. Moreover, for every a € K(x) [Dx] 
we can find a basis of the K-vector space V(a) ME, where E is the field of all Liou- 
villian functions in x [181, 168, 68]. For a recurrence operator a € K(x)[S;,], there 
are algorithms for finding all its hypergeometric solutions (see Section 15.4 below) 
or d’Alembertian solutions (see Section 15.4.2 below). There are also algorithms for 
solving recurrence equations in terms of hypergeometric series [54], or for solving 
differential equations in terms of special functions [47, 52]. 
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Only in very special circumstances will an Ore operator have closed form solu- 
tions. More often, there will be no better way to express the solutions of an operator 
than the operator itself, plus possibly some finite amount of additional data. For ex- 
ample, a sequence solution (f,)*°_9 of a recurrence operator a € K[x][S;] is uniquely 
determined by the terms fo, fi,..-,fr—1,fé,,fé>---»fé,, where r is the order of the 
operator and &; —r,...,&4—r are the positive integer roots of the leading coefficient 
of a, which is an element of Kx]. In general, when there are positive integer roots, 
not every choice of initial values gives rise to a sequence solution. On the other hand, 
if there are no positive integer roots, then the sequence solutions (f;,)*°_) of a are in 
one-to-one-correspondence with their truncated versions ( ree 


Example 15.2.14 (Initial Values) = /. Consider the recurrence operator a := 
(n +2)S? — (2n + 3)Sn +(n +1) € Q{n|[S,]. In this case, the leading coeffi- 
cient (n+ 2) € Q[n] has no positive integer roots. Therefore the solution space 
V(a) C QN has dimension two. For any two numbers u,v € Q there exists pre- 
cisely one sequence (fn) € V(a) such that fo =u and f| =v. For example, 
for the choice u=0,v = 1 we get the harmonic numbers Hy = Yi_, i 


2. Consider the recurrence operator a:= (n—6)(n—2)(n? — 10n +29)S? — (n* — 
9n + 24) (n* — 5n — 2)S, + 2(n —5)(n? —8n +20) € QnJ|S,]. This operator 
has order r = 2 and its leading coefficient has d = 2 positive integer roots. It 
follows that r=2 < dimgV(a) <4=r-+d. In fact, the dimension is 3, and 
the sequence solutions starting as follows form a basis: 


e 10. 3 Sted fe 26 176 
1, 1’ 0, 0, T05’ 315” 


21? 1’ 


7? 7. 
4 26 176 
Of 808 SOs °105. “de Be. Fe 50; ee es 
0, 0, 


0, 0, , 84 FO OF IL 3 6 


We can let differential operators a € K|x][D,] act on the ring M = K|[x]] of formal 
power series. If a has order r and the coefficient of D? in a (which is an element 
of K[x]) does not contain x as a factor, then a has r linearly independent solutions 
in M, and there is a basis of the solution space V(a) that has the form 


10h Ox ee Oe Ox (ee ee 
Oe Oe tebe 0 See et 
O+0x+0x7 +---+ 0x7? + 1x! + Ox" eu 


where the symbols LJ stand for certain elements of K that are determined by the 
operator a. 

In general, if x is a factor of the leading coefficient of a, the dimension of V (a) 
in K|[x]] may be smaller than the order r. In any case, it is possible to construct r 
linearly independent generalized series solutions of the form 


exp(p(x'/*))x%q(x!/*, log(x)) 


Computer Algebra 995 


where s € N\ {0}, p is a polynomial of degree < 5 with p(0) = 0 and coefficients 
in K, the algebraic closure of K, a@ € K, and q = q(x,y) € K[|[x]][] is a polynomial 
in y with coefficients that are formal power series in x. See [88, 181, 102] for details 
on this construction. An analogous result for the shift case says that any operator a = 
do +-+-+a,S', € K[n][S,] with ag 4 0 #.a, admits r linearly independent generalized 
series solutions of the form 


(=) "orexp(p(n!/*))n%q(n-"*,tog(n)) 


where e = exp(1), y€ Q, @ EK, pis a polynomial of degree < s with p(0) =0 and 
coefficients in K, a € K, and q = q(x,y) € K|[x]|[y]. This construction is explained, 
for example, in [187]. 

In the first place, generalized series solutions are only formal solutions. However, 
in typical examples arising in combinatorics, they can be interpreted as asymptotic 
expansions of function or sequence solutions, as illustrated in the following example. 


Example 15.2.15 (Asymptotic Solutions) The operator (n + 2)S% — (4n +2)S, + 
(n+1) € Q[nl[S,] has a unique sequence solution (fn)? with fo =1, fi =2. It also 
has two generalized series solutions, the first terms of which are 


-1/2 3473-1, 2-3-2 , —15410/3.-3 , 133-7073, —4 
(2+V3)"n ae pen Wan thy oT En ) 


_ ./a\ny-1/2 (7 4. -3-v3 ,-1 4 V3, -2 | -15-10V3 ,-3 | 133+70V3,,-4 4, 
(2—v3)"n (1 iz 7 Tae 92? 1536” 


We can expect the asymptotic behavior of the sequence solution (fy); to be de- 
scribed by a certain linear combination of these two series solutions. Indeed, be- 
cause of |2+ V3 | > |\2- V3\, the first series dominates the second, so it is plausible 
to expect that we have 


fa =C(2+ V3)" n 1? (1 | 3B y 11 2-VB3q ae 1S+10V3 34.0(n >) 


as n— ©, for a certain constant c € R. In order to get a feeling for c, consider the 
quotient sequence 


Sn 
(24+ -f3)en-1/2 (1 Se irl 25Y3n-2 + =I 3) 


n= 


By direct calculation, we find that 


U9 © 0.58560385580375675584 
u100 © 0.58560331421465401061 
u1000 © 0.5856033 1416909798511 
U10000 & 0.585603314169093496 14. 


These figures provide convincing evidence (though not a formal proof) that the limit 
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c = limy40Uy, exists, and that its value is approximately 0.5856033 14169093496. 
Using lattice reduction as explained in Example 15.2.4, we may be able to derive 
from this approximation a plausible conjecture for an exact formula for c. In this 


example, we may find c ne 4/ bese 


15.3 Counting algorithms 


Combinatorial algorithms can be divided into three categories. Algorithms from the 
first category are used for generating combinatorial objects, either all the objects 
from a particular class, or a random element according to some prescribed probability 
distribution. Algorithms from the second category are used for searching objects 
with special properties within a class of objects, for instance an element for which a 
certain objective function attains an optimal value. The third category are algorithms 
for counting the objects of a certain class, typically as a function of a parameter 
measuring the “size” of the objects in the class. 

In this chapter we mostly discuss counting algorithms, as these are more closely 
related to computer algebra than the other two types. For encyclopedic overviews 
on generation and search algorithms, see [138, 116, 107] or the appropriate other 
chapters of this handbook. 


15.3.1 Special purpose algorithms 


When there is an algorithm for generating all the objects from a certain family, say of 
a given size n, then there is obviously also an algorithm for counting them. But such 
algorithms are usually prohibitively expensive: Their runtime is at least proportional 
to the number a, of objects, which is typically at least exponential in n. In this section 
we give an (unsystematic and incomplete) selection of examples for how faster algo- 
rithms can be obtained. The algorithms discussed in this section are only elementary 
applications of computer algebra using no more than arbitrary precision arithmetic 
or computations with truncated formal power series. Sometimes the construction of 
such algorithms is straightforward. 


Example 15.3.1 (Stirling Numbers) Let S2(n,k) be the Stirling numbers of the sec- 
ond kind, defined as the number of partitions of a set of size n into k nonempty disjoint 
subsets. It follows easily from the combinatorial definition that Sy(n,k) satisfies the 
recurrence equation 


So(n,k) = So(n —1,k — 1) +kSp(n—1,k) 


for alln,k > 1. Together with the initial values Sy(0,0) = 1 and S3(i,0) = S2(0,i) =0 
for i>, this recurrence gives rise to an algorithm for computing Sx(n,k) for any 
givenn,k > 1. This algorithm will be reasonably efficient if it is implemented in such 
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a way that no term S(i, j) is computed more than once. For example, to compute the 
term S>(n,k), set up an array S of size (n+1) x (k+1), initialize S[0,0] = 1, S[i,0] = 
0 fori=1,...,n, S[0, j] =O for j =1,...,k, and then for i=1,...,nand j =1,...,k 
set Sli, j] = S[i— 1, 7-1] + jS[i-—1, j]. Finally return the value S|n,k]. This routine 
obviously requires only nk operations in Z while a naive recursive implementation 
of the recurrence would run in exponential time because of excessive recalculation. 


It is typical that the number of objects in some combinatorial family can be com- 
puted using some sort of recursion. The recursion is however not always as simple, 
or as easy to see, as in the example above. Sometimes the key to an efficient counting 
algorithm is a recurrence for an associated object, from which the number of interest 
can be deduced in a second step. 


Example 15.3.2 (Restricted Lattice Walks) Consider the lattice walks in N* C Z? 
starting at the origin (0,0), consisting of exactly n steps, each of which is one from 
{(0,—1),(—1,0), 1, 1) }. Such walks are called Kreweras walks [117, 40, 41]. How 
many Kreweras walks with n steps are there? The combinatorial description gives 
rise to a multivariate recurrence for the numbers an,j,; counting the number of Krew- 
eras walks with n steps and end point (i, j): 


On,i,j = An—1,i41,j) + M-1,i,j+1 + An-1,i-1,j-1- 


Together with the initial values ago = 1 and agj,; =0 ifi #0 or j #0 and the 
boundary conditions aj,—1,0 = Gno,-1 = 0 for all n, the recurrence determines the 
numbers ani; (n,i, j € N) uniquely and can be used to compute them. For example, 
we find a9 4,5 = 160. The number ay of all Kreweras walks with n steps (and ar- 
bitrary end point) is then obtained via ay = Yj~9 Lj—0 4nji,j = ar Li-o y,j,j- The 
sequence Starts with 1, 1, 3, 7, 17, 47, 125, 333, 939, 2597, 7183, ... (A151265 in the 
OEIS [170]). 


Recurrence equations are not always as nice and natural as in the previous two 
examples, and they do not always give rise to efficient algorithms. An example for 
a family of combinatorial objects for which even the construction of “slow” recur- 
rences is nontrivial are 1324-avoiding permutations [130, 91]. Due to the lack of bet- 
ter counting algorithms, only the first 31 terms of the counting sequence are known 
so far. 

Instead of a recurrence for a counting sequence (a,)*_, it is sometimes eas- 
ier to come up with a functional equation for its (ordinary) generating function 
a(x) := LY" _panx" € Q|[x]]. Such functional equations can also give rise to efficient 
algorithms for calculating ay. 


Example 15.3.3 (Unlabeled Rooted Trees) Let a, be the number of unlabeled 
rooted trees. There is no obvious recurrence for ay, but it can be shown (e.g., [107, 
Section 7.2.1.6]) that the generating function a(x) = V7; anx" satisfies the func- 
tional equation 


a(x) =x exp(a(x) + Fa(x) + fa(x?) fe), 


998 Handbook of Enumerative Combinatorics 


This equation has a unique formal power series solution, and its coefficients can 
be calculated recursively. For, if the coefficients a, ,a2,...,an—, are known, then we 
can calculate the nth coefficient ay = |x"|a(x) using the formula that is obtained by 
picking the coefficient of x” from the right-hand side: 


An = [x"|x exp(a(x) + Fa(x") 4+) 


"~Yexp(a(x) + 3a(x7) +--+) 


Note that the truncation in the last step is correct because |x°]a(x) = 0 implies that 
none of the discarded terms has a chance to contribute to the coefficient of x"~'. 
Note also that for every fixed n, the final expression involves only a finite number of 
operations, and only depends on the terms a\,a2,...,@,—1 computed earlier. 

The resulting sequence starting with 1, 1, 2, 4, 9, 20, 48, 115, 286, ... (AO00081). 


Another fruitful source of counting algorithms is the transfer matrix method, 
which can be used to derive formulas for generating functions. We give here only 
an example application. A general introduction can be found in [172, Section 4.7]; 
for implementations and additional examples see [131, 133, 193]. 


Example 15.3.4 (Tilings) We want to tile a rectangle of size n x 3 without holes 
using stones of shape | x 1 and 1 x 2 (possibly rotated by 90 degrees). In order to 
determine the number of different tilings, consider the following matrix, that encodes 
the possible transitions from one partial tiling to another by placing a stone at the 
lowest and leftmost free position. The variables x, and x2 correspond to placing a 
stone of size I or 2, respectively, and the variable z keeps track of the total height of 
the construction. For example, the entry x2z at position (3,6) means that a partial 
tiling with a “skyline” of type a.can be transformed into a partial tiling with a skyline 
of type « by placing a stone of size 2 (vertically) in the middle position. This enlarges 
the overall height of the tower by one, which is reflected by the multiplication by z. 


a a D o ao m a 1 Bo c 

= 0 x2 O x %x x x O DO 0 0 x 
a | xz O 0) O xz O O x x; x,y O O 
he 0 0 0 0 0 »z 0 0 0 0 x O 
a 0 0 0 0 0 0 mz 0 0 0 0 0 
= 0 0 0 0 0 0 0 0 0 0 x» O 
a5 0 0 0 x» O 0 0 0 x» 0 0 O 
a] XZ XxX xX 0) 0 0) 0 0 0 x» O x 
ae Wee. OS Oe Os OOS. SO OS 20 0. 20% oD 
fh 0 0 xz O 0 0 0 “4 O 0 0 O 
An 0 0 0 xz O 0 0 0 0 0 0 0 
A 0 mz O 0 0 0 0 0 0 0 0 +0 

0 0 0 0 0 0 0 «7» O0O 0 0 O 
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IfA denotes this matrix, then the entry at (1,1) of the matrix power A* is a polynomial 
in X1,X2,z that encodes all the possibilities to tile a rectangle with exactly k stones. 
A monomial Cee in this polynomial means that there are c different ways to fill 
a3xn rectangle with exactly e, stones of size 1 and exactly e2 =k — e, stones of 
size 2. The generating function for the number of all tilings (without regard to the 
number or type of stones) is given by EA iletee Its first terms are 1, 3, 
22, 131, 823, 5096, ... (A033506). 


Finally, there are situations where a counting algorithm can be obtained from a 
result about the asymptotic behavior of the counting sequence. 


Example 15.3.5 (Partition Numbers) Let p(n) be the number of ways to write the 
integer n as a sum of positive integers (regardless of the order of the summands). The 
sequence starts with 1, 1, 2, 3, 5, 7, 11, 15, 22, 30, 42, ... (A000041) 

One way to compute these numbers is via the pentagonal number theorem of 
Euler, which states that 


pin) = (1) 'p(n—2k(3k—-1)) (n> 1). 
k 


This recurrence, together with the initial values p(0) = 1 and p(n) =0 forn <0, can 
be used to calculate p(n) for any positive integer n. 

A more efficient algorithm can be derived from the Hardy-Ramanujan- 
Rademacher formula, which in its simplest form amounts to the asymptotic estimate 
p(n) ~ aia exp(%/2n/3) (n — ©) and in its full form is a convergent series with 
an explicit error term that is so accurate that it can be used to approximate the value 
of p(n) to a guaranteed error of less than 1/2. Then rounding to the nearest integer 
gives the exact value of p(n). See [90] for details. 


The examples in this section have in common that algorithms for counting certain 
combinatorial objects were obtained as a result of applying some counting theorem 
about the objects. It would be easily possible to give many further examples of this 
type, as nearly every theorem in combinatorics can be rephrased as an algorithm for 
counting objects, even if the theorems themselves were derived by hand. We will 
now discuss how to apply computer algebra in a more essential way, using it not only 
for implementing combinatorial theorems but for also for their derivation. 


15.3.2 Combinatorial species 


Recursive definitions of combinatorial objects can often be translated systematically 
into recurrence equations for the corresponding counting functions. The theory of 
combinatorial species [92, 23] provides a uniform and systematic framework for such 
constructions, and gives rise to algorithms for automatically translating recursive 
specifications of classes of combinatorial objects into functional equations for the 
corresponding generating functions, which can then be processed further by other 
algorithms. 


1000 Handbook of Enumerative Combinatorics 


15.3.2.1 Formal definition and associated series 


A combinatorial species is simply defined as a functor from the category of finite sets 
with bijections into itself. For example, the species DAG of directed acyclic graphs 
maps every finite set V to the set DAG(V) of all directed acyclic graphs with vertex 
set V, and it maps a bijection o: V — W into the bijection DAG(o): DAG(V) > 
DaAG(W) that relabels the vertices according to o. Observe that DAG can be regarded 
as a formal description of the concept of directed acyclic graphs that abstracts away 
from the type of objects used as vertices. 

If F is a combinatorial species, then the cardinality of F(V) only depends on the 
cardinality of V, not on the particular elements of V. We can therefore ask in general 
for the number a,, of different objects F produces when applied to a set V of cardi- 
nality n. We define dp (x) := L_o|F({1,.-.,n})|*; as the exponential generating 
function for F, so that, for example [x>]@pac (x) = 20981 means that there are 29281 
different directed acyclic graphs with five distinct fades 

For counting unlabeled structures, define an equivalence relation on F(V) by 
setting p ~ q for p,q € F(V) if there exists a bijection o: V — V such that 
F(o)(p) =q. The equivalence classes of ~ can be interpreted as the unlabeled coun- 
terparts of the objects in F(V). We define the (ordinary) generating function for F 
as ap (x) = DP o|F({1,...,2})/~ |x". For example, [x°]apac(x) = 302 means that 
there are 302 different directed acyclic graphs with five indistinguishable nodes. 

We are primarily interested in the ordinary and the exponential generating func- 
tions associated to a species F’, but it turns out that an additional series is needed for 
computing them. Every bijection o: V — V is a permutation of the elements of V, 
which can be written uniquely in terms of cycles. Write o; for the number of cycles 
of length i in this representation. For example, oj is the number of fixed points of o. 
Then the cycle index series of F is defined as 


Zr (x1,X2,- =Da =( y F(a) aba) € Ql[ai,x2,...]], 


TESn 


where S,, denotes the symmetric group, i.e., the set of bijections of {1,...,n}, and 
F(z) is the number of fixed points of the bijection F (7) from F ({1,...,7}) to itself. 
We have d(x) = Zr (x,0,0,...) and ag (x) = Zr(x,x7,x°,...). See [23] for examples 
and a more elaborate discussion. 


15.3.2.2 Standard constructions 


In general it is not obvious how to determine the cycle index series for a species F. 
However, many species of interest (and the corresponding series) can be constructed 
starting from a small number of elementary species (and the corresponding series) 
using a small number of operations on species. The most important constructions are 
summarized below. 


e Sum. Disjoint union of F-objects and G-objects: 


(F +G)(V) := F(V) UG(V) = (F(V) x {e}) U(G(V) x {o}). 
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e Product. All pairs (f,g) where f is an F-object and g is a G-object: 


(FG)(V) := U (FU) x GV \U)) 
UCV 
One part of the elements of V is used for labeling f, the remaining elements 
are used for labeling g. 


e Substitution. F-objects labeled by G-objects: 


(FoG)(V):= (J F({I,...,m}) x G(V1) x G(V2) x + x G(Vin), 
V1 ,V2,---Vin 


where the union is taken over all set partitions of V into non-empty disjoint 
subsets V = Vj U V2 U-+» U Vn. 

e Pointing. All F-objects in which one of the labels is singled out: OF (V) := 
F(V) x V. For example, if T is the species of trees, then OT is the species of 
rooted trees. 

e Restriction. For a species F and an integer k, define F_, by F_,(U) :=U if 
|U| =k and F_,(U) = 9 otherwise. The restrictions F<, and F, are defined 
analogously. 


Typical starting points for constructions are simple species such as the 0-species 
with 0(V) = 0 for all V, the 1-species with 1(0) = {0} and 1(V) = 9 for all V £9, or 
the singleton species X with X(V) =V for all V with |V| = 1 and X(V) = 0 otherwise. 

Further useful building blocks of a construction are the species of sets, defined 
by SET(V) = {V} for all V, the power set species PSET(V) := {U: U CV}, and the 
species of cycles, defined as 


Crc({v1,.--,¥m}) = { (V(t) >++++Va(m)) :©ESm}/_, 


where S,,, is the symmetric group and ~ the equivalence relation that identifies two 
tuples if and only if they are cyclic permutations of one another. 

The cycle index series for these series are known explicitly, and it is known how 
each of the operations above affects the cycle index series. This knowledge can be 
encoded into algorithms that produce for a given species specification a lazy series 
representation of the corresponding generating functions. 


Example 15.3.6 (Direct Constructions) The singleton species X can be viewed as 
representing an atomic object, e.g., 0. For fixed positive k € N, the objects in the 
species X* := XX ---X (k factors) can be viewed as chains of exactly k atoms, i.e., 
something like o-o— --- —o. The objects in X +X* +X? +X* are then chains con- 
sisting of at least one and at most four nodes. The pointing operation singles out 
one of the nodes as special, so an object in @(X +X? +X? + X*) can be viewed 
as something like o-e—-o—o or o—c—e—o, for example. Next, consider the species 
S=CyYco@(X +X? +X? +X°*). Its objects can be thought of as graphs that are 
constructed by cyclically linking together a finite number of chains with at most four 
nodes, the link being attached to one of the nodes, for example 
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O. 
a ore 
i 
‘o 
The first terms of the generating functions s(x) and 5(x) for S can be computed di- 
rectly from the defining expression S stated above, the result being 


s(x) =x+3x7 + 8x3 +21x4 450x° + 128x°+4--- 
and 5(x) =x+4x? + 10x? + 26x4 +54x° + 144x°4.---, 


respectively. 


15.3.2.3. Recursive specifications 


Recursive definitions of combinatorial structures can be formulated as equations for 
combinatorial species in terms of the constructions introduced above. For example, a 
(finite) sequence is by definition either the empty sequence or an object followed by 
a sequence. In the language of species this definition can be formulated as equation 
SEQ = 1+X SEQ. The (unique) species solution SEQ of this equation is the species 
of finite sequences. 

More generally, we can consider systems of equations, consisting of several equa- 
tions and several unknown species. In general it is not obvious whether a given sys- 
tem has a solution, or whether its solution is unique. For systems composed of sum, 
product, substitution, restriction, 1, X, SET, SEQ, and CYC, Pivoteau, Salvy and So- 
ria [148] introduce the notion of well-foundedness, and they prove that every well- 
founded system has a unique solution. This result generalizes the implicit species 
theorem of Joyal [92, 23], which in turn generalizes the implicit function theorem 
for formal power series (Section 15.2.3). It can be decided algorithmically whether a 
system is well-founded, and in this case, there are algorithms for efficiently comput- 
ing the first terms of the generating functions, for drawing uniform random elements 
of prescribed size, and for numerically determining the asymptotic behavior of the 
counting sequences [148, 146, 147, 69, 118, 119]. 


Example 15.3.7 (Funny Trees) Consider the (artificial) family of trees in which 
nodes are either black or white, the order of the subtrees of a white node matters 
but the order of the subtrees of a black node does not matter, and white nodes cannot 
have white children. The species T of such trees may be defined by the following sys- 
tem, with two auxiliary variables W and B representing the species of trees with white 
or black roots, respectively, and Xw and Xp two distinguishable singleton species: 


T=W+B, W=Xyw(SEQOB), B=Xpg(SETOT). 


Starting from this specification, the first terms of the counting sequence can be de- 
termined automatically, the result being 0, 2, 3, 9, 30, 110, 423,... 


Computer Algebra 1003 


More can be done for more restricted classes of systems. Of particular interest are 
systems composed only from sum, product, 1, X, SEQ, SEQ_;, SEQ<, and SEQs, 
(k a fixed integer). Flajolet and Sedgewick [75, Sect. I.5.4.] show that the solution 
species of such systems always have algebraic generating functions. (Their “sym- 
bolic method” is a variant of the theory of species, see [148] for a detailed compari- 
son.) The system of species equations can be automatically translated into a system 
of polynomial equations for the generating functions corresponding to the species 
variables, and Grébner bases (Section 15.2.2) can be used to obtain from this sys- 
tem annihilating polynomials for the generating functions of each of the individual 
species. 


Example 15.3.8 (Planar Trees) The species TREE of nonempty planar trees is the 
solution of the equation TREE = X (SEQo TREE). Taking this equation as input, it 
can be found algorithmically that the ordinary generating function T (x) of nonempty 
planar trees satisfies the equation T (x)? — T (x) +x =0. 


15.3.3 Partition analysis 


The sequence p(n) counting all the integer partitions was already mentioned in Ex- 
ample 15.3.5. In this section we discuss algorithms for counting generalized classes 
of partitions, or partitions subject to certain constraints. It turns out that the counting 
sequences of many interesting classes of partitions are much simpler than p(n). Sim- 
ple expressions for their generating functions can often be obtained algorithmically. 


15.3.3.1 ©-Calculus 


At the end of the 19th century, MacMahon [125, 126, 127] considered classes of 
partitions whose generating functions can be expressed as the positive part (cf. Sec- 
tion 15.2.3) of a simple power series. For example, the generating function counting 
all the partitions of n into at most three parts can be written in the form 


(wear y Le) 


iy ,i2,13=0 


€ Qi]. 


Ay=Ap=1 


The series to which the operator [A;~A; is applied is a multivariate Laurent series 
expansion of the rational function 


1 


— € Qi, 2,4). 
(1—Ajx)(1—A,A, “x)(1—Az *x) 
MacMahon reserves the names 4;,A2,... for variables to be eliminated by positive 
part extraction, and he uses the notation Q for the operator ([A7 Ay ..-]-) |ay=ay—-<1- 
2 


The variables A; are called slack variables. 
Expressions involving the Omega operator can be simplified using general 
rewrite rules such as 
Cc Cc 


OQ Gap =B/) ~ TAB) 
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where A,B,C are free of A. By applying this rule twice (once for eliminating A,, and 
then once more for eliminating also Az), we find 


Q 1 1 
> (1—Ayx)(1- Aap 4x) (1 — Ag) 


In a series of articles [10, 12, 11, 9, etc.], Andrews, Paule, and Riese have turned 
this approach into an algorithm. The algorithm is applicable to multivariate rational 
functions whose denominator factors into binomials 1 — A;''---Ajtmx}! ---x;* for cer- 
tain fixed integers u1,...,W¢m,V1,---,Vg- The result is a rational function in x),...,x,. 
An alternative algorithm for solving the same problem, also based on MacMahon’s 
ideas, was proposed by Xin [188]. 

Rational functions of the required form arise naturally from geometric series in 
Ay,--+;Am and x1,...,x; in which the exponents are integer-linear combinations of 
the summation variables. The exponents of the x; carry the integers whose partitions 
are to be counted, and the exponents of the A; express inequality constraints by which 
the type of partitions to be counted is defined. The approach is limited to problems 
that can be formulated in terms of a finite number of A; and x;. 


Example 15.3.9 (Plane Partitions) A plane partition of nis a matrix of nonnegative 
integers whose entries sum to n and where the entries are non-increasing along rows 
(from left to right) and columns (from top to bottom). Let a, be the number of plane 
partitions of n of size 2 x 2, and let a(x) = Vp anx" be its generating function. In 
order to obtain a rational expression for a(x), apply the partition analysis algorithm 
to 


a 
ty1—12 9 titan 4 12-122 4 i21 122 i +i12 +21 +122 
» Ai Ay As hg % 


i115412,121 422=0 


1 
“GERAD d= as Nada ae ee. 


. 1 
The result is (9 (012)2(13)° 

It is also possible to compute a generating function that actually “gener- 
ates” all the plane partitions of size 2 x 2, by applying the algorithm to the se- 
ries in Ay,Az,A3,A4,x11,%12,X21,%22 obtained from the series above by replacing 


141 +142 +191 +i22 411 5.112 21 ,.122 . . fi 2 
x by X4'}.X15%X31X39- The resulting rational function is then 


1 —xt,x12%21 
(12x11) (1 —x11x12) (1 — x x21) (1 — x1 e2x21) (1 — 01101221422) 


Its multivariate series expansion is the sum of all terms x} x'!3x3|x353 for which 
(2 i2 


as 2) is a plane partition of size 2 x 2. 


Computer Algebra 1005 


Not only inequalities but also equational constraints can be handled by the Omega 
calculus. Because A = 0 if and only if A >O/AA <0, such constraints can be encoded 
using two slack variables whose exponents differ by a factor of —1. While this is fine 
in theory, implementations may handle equational constraints in a more efficient way. 
MacMahon writes Q for the operator [APAY---]. 


For extracting the positive part or the constant coefficient from a series that is not 
of the form required for Omega calculus but is still holonomic, see Section 15.4.3. 


15.3.3.2 Ehrhart theory 


Partition analysis is closely related to discrete geometry. The objects of interest in 
discrete geometry are areas in a finite-dimensional real space defined as intersec- 
tions of finitely many half spaces (viz., solutions of systems of finitely many linear 
inequalities). One of the questions of interest is to count the number of points with 
integer coordinates in such areas. 

For a bounded set P C R", the Ehrhart function [71] is defined as 


Ehrp: NN, Ehrp(d) := |dP NZ", 


where dP := {dx : x € P} refers to the d-fold dilate of P. Observe that Ehrp(1) is the 
number of integer points in P. 

A set P C R” is called a rational polytope if it is the convex hull of finitely 
many points in Q”. This is the case if and only if P is bounded and there is a ma- 
trix A € Z”*" and a vector b € Z”™ such that P = {x € R": Ax < b}, where “<” is 
meant component-wise. In this case, Ehrp(d) is a quasi-polynomial, i.e., its generat- 
ing function £7) Ehrp(d)x‘ is a rational function whose denominator is a product of 
cyclotomic polynomials. Among the remarkable features of the Ehrhart function for 
rational polytopes is the Ehrhart-Macdonald reciprocity [124, 20], which states that 
Ehrp:(d) = (—1)" Ehrp(—d), where P° denotes the interior of P. (Note that P itself 
is a closed set.) 

A rational polytope P is called a lattice polytope if it is the convex hull of finitely 
many points in Z”. For such polytopes, the Ehrhart function is a polynomial in d, the 
Ehrhart polynomial of the polytope. 


Example 15.3.10 (Birkhoff Polytopes) For fixed n € N, let B, C R"" be the set of 
all matrices A = ((4i,j))7 j= with aj; > Oand Y"_, aj; = 1 forall j and Li-l aij=1 
for all i. Reading matrices as points in R”, the set B, is a polytope, called the n-th 
Birkhoff polytope. Beck and Pixton [19] have determined the Ehrhart polynomials of 
B, forn =1,...,9. While it is not hard to find Ehrg, (d) = 1, Ehrg,(d) =d +1, and 
perhaps Ehrg,(d) = gd? + a3 + ea + zd +1, the computational cost increases 
rapidly with n. For the calculation of Ehrg,(d), they report a computation time of 
324 CPU days on a 1.0 GHz Linux machine. Its degree is 64, and the leading coef- 
ficient is a rational number with a numerator consisting of 45 decimal digits and a 
denominator consisting of 85 decimal digits. 
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Let P C R” be a rational polytope with vertices v1,...,Vm € Q”. Its dilates dP 
(d > 0) can be viewed as slices of the cone C C R"*! generated by {v1,...,Vm} x {1}, 
Le., C = {Av +--+ +Amim 2 A1,.--,4m > 0} where the generator vectors 7; are 
obtained from the v; by appending a coordinate 1. The complete generating function a 
of such a cone C is defined as the formal sum of all the monomials ote . Sp ae with 


n+l 
(e1,.-.,€n41) €CNZ"*!, It lives in the ring Zc[[x1,...,xn+1]] (cf. Section 15.2.3; 
note that C is a pointed cone). Setting x) = --- =x, = 1 and x,4; =x ina yields the 


generating function )7_. Ehrp(d )x4 of the Ehrhart polynomial for P. 

The generating function a is a multivariate series expansion of a certain rational 
function in x),...,Xn,41. It is possible to compute the numerator and denominator of 
this rational function given as input the basis vectors ¥),...,Vm. After scaling the v; 
by an appropriate positive integer, if necessary, we may assume that all the v; belong 
to Z"*!. For simplicity, let us further assume that the ¥; are linearly independent 
in R"*! (see [20] for a discussion of the general case). Writing 7; = (Vi, wes Vint1)s 
we then have 


ey en ent+l 

pn Vette ETUC) AY Fn Fn 

a m Vit Vintl ? 
play" Xe) 


where II(C) := {A101} +--- +Am¥in 10 < A1,...,Am < 1} is the fundamental par- 
allelepiped of C. As II(C) is a bounded set, it contains at most finitely many integer 
points. 

The numerator in this expression is problematic because the number of monomi- 
als may be quite large. Indeed, writing down the numerator expression may consume 
exponentially more space than writing down the input basis ¥,..., 1, even if we fix 
the dimension n of the ambient space. It is therefore impossible to compute this ra- 
tional function representation efficiently. However, there do exist cheaper represen- 
tations: Barvinok [16] showed that for every cone C C R”*! generated by linearly 
independent vectors 7,,...,, € Z"*! there exist integers L, e,,...,ez and vectors 
Up = (Urs s2s,Uene) and wee = (Weiss Wein) = 1,.4,L, 0= 1,...,n4+1) 
such that 


L , Ue Ul n+1 

= (—1)*xp ae 

— +1 Wei, Wein+l 
rie ra ar le ee) 


and the space needed to store this representation grows at most polynomially with the 
space needed to store the basis ¥1,...,V, provided that n is fixed. He also provides 
a polynomial time algorithm for computing such a representation. 

Barvinok’s idea is thus to write the large rational function a as a linear combi- 
nation of small rational functions. Bringing his representation on a common denom- 
inator is not a good idea: it will lead to the exponentially longer expression stated 
earlier. Instead, one should try to extract the desired information directly from his 
representation. For example, to obtain the number of integer points in the rational 
polytope P C R”, compute the Barvinok representation of the complete generating 
function a of the associated pointed cone C C R"*! setxy =--- =x, = land Xn4+1 =x 
in this expression, extract the coefficient of x! of the series expansions of each sum- 
mand individually, and add up the results. See [17, 122, 44] for further information. 
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15.3.4 Computational group theory 


Groups arise in combinatorics for various reasons. First, they may themselves be 
a source of counting or enumeration questions (How many elements does a group 
have? How many subgroups? How many groups of a particular type are there?, etc.). 
Secondly, and perhaps more importantly, groups are used for describing symmetries 
in combinatorial objects. For example, counting the number of ways to color the faces 
of a cube with three different colors requires some knowledge about the symmetry 
group of the cube. Similarly, a functional equation for the generating function of 
ternary trees can be obtained easily from a functional equation for the generating 
function of planar ternary trees by letting the symmetric group $3 act on the subtrees 
of each internal node. 

Details about such applications of groups in combinatorics are explained in other 
chapters of this handbook. In this section, we only give a short summary on how 
groups may be represented in a computer, and some algorithms for answering ques- 
tions about groups. For further reading, we recommend the lecture notes [87] and the 
textbooks [167, 164, 86]. 


15.3.4.1 Permutation groups 


If G is a group, a set B C G is called a generating set if every element g € G can 
be written (not necessarily uniquely) as a finite product of elements of B. Trivially, 
G itself is always a generating set for G, but most groups G admit generating sets B 
that are much smaller than G. Every finite group G is isomorphic to a subgroup of 
a symmetric group S,, and therefore, every finite group G can be represented by a 
set B of permutations. 

The main algorithm for subgroups G of S,, specified via a generating set B is the 
Schreier-Sims algorithm [166, 164, 87]. It constructs an alternative representation 
of G, called a base and strong generating set (BSGS), which makes it easy to 
answer a number of fundamental questions about G. From an application point of 
view, this is similar in spirit to the use of Grébner bases for polynomial ideals, with 
Sn playing the role of the polynomial ring, G the role of an ideal, B the role of a 
given basis, and the BSGS for G playing the role of a Grobner basis of the ideal. 
Computationally however, the algorithms are not related. 

Some of the problems that can be solved using the Schreier-Sims algorithm (or 
related algorithms) for a group G < S, given via a generating set B C G are the 
following: 


e Compute the group order |G]. 

e Construct a bijection between {1,...,|G|} and G. 

e Given x € {1,...,n}, compute its stabilizer G, := {g € G: g-x =x} <G. 
e Given x € {1,...,n}, compute its orbit Gx = {g-x: g © G} C {I,...,n}. 

e Givenh € S,, decide whether h € G. 

e Given h1,h2 € Sy, decide whether there exists g € G with h; = g'hog. 

e Given H <G, compute the smallest normal subgroup of G containing H. 
e Given H <G, construct a set of representatives of G/H. 
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e Decide whether G is solvable, or nilpotent. 

e Construct a vector B = (bj,...,bm) of pairwise different numbers b; € 
{1,...,n} such that the only element of G fixing each 5; is the identity, and a 
set S C G such that for all i= 1,...,m we have (SANG) = GM, where 


G) :={gEG|Vi<i:g-bj =b;)} <G. 


The last item contains the definition of a base (the vector B) and strong generating 
set (the set S). 


Example 15.3.11 (Schreier-Sims Algorithm) Consider the group 
G = ((1,2)(3,4), (1,2,3)(8, 10,9), (1,6,7,5)(8, 10)) < Sto. 


A possible output of the Schreier-Sims algorithm applied to G is the base B = 
(10,5,2,3,4,7, 1) and the strong generating set 


S = {(6,7)(8,9), (4,7,6), (3,4)(6,7), (2,3,5)(8,9, 10), 
(2,5)(3,4), (2,7)(3,4), (1,6,7,5)(8, 10), (1,6)(8,9)}. 


Once this data is known, it requires only some inexpensive calculations to de- 
rive that |G| = 15120, that the two orbits of the natural action of G on 
{1,..., 10} are {1,...,7} and {8,9,10}, that the stabilizer of x = 8 is the group 
((1,5)(9,10), (1,6)(9,10), (1,4,6), (1,6)(3,4), (2,6)(3,4), (2,7)(3,4)), that the 
permutation (1,2,3,4,5) belongs to G and the permutation (1,2,3,4) does not, 
that the smallest normal subgroup of G containing (1,2,3,4,5) is the group 
((1,2,3,4,5), (1,4,3,5,2), (1,3,6), (2,3,6,7,4)), and so on. 


15.3.4.2 Finitely presented groups 


Let Q be a set. A word over © is an element of Q* := U7) Q”. The set Q* 
together with the concatenation (a1,...,dn)(b1,...,0m) = (a1,---,@n,b1,.--,Dm) 
is a monoid, called the free monoid over Q. The neutral element is the empty 
word € := () € Q° C Q*. It is customary to write a word (a1,a2,...,dn) simply 
aS €1a2°::ay. 

Let Q7! :={a-!: a@ € Q} bea disjoint copy of Q, and let F C (QUQ7!)* be the 
set of all words that contain no subwords of the form a~!a@ or aa~!. The operation 
o: F x F — F is defined as concatenation followed by repeated removal of subwords 
of the form a~!a or aa! until no such subwords remain. Then (F,0) is a group, 
called the free group over Q. 

Let now R C F, and write (R) for the smallest normal subgroup of F containing 
all the elements of R. Then G = F’/(R)r is a group. If Q and R are finite, we call 
G a finitely presented group. The common notation is G = (Q | R). For example, 
we have (a | a>) = Z/5Z (easy), (a,b | a?,b?, (ab)*) = S3 (less obvious), and (a,b | 
a~'b~'ab) = Z? (easy). Every finite group is finitely presented but not every finitely 
presented group is finite. 
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Finite presentations are a powerful tool for representing complicated groups by a 
finite (and often small) amount of data. However, there are not too many algorithms 
for solving problems about groups specified in this way. Many interesting problems 
turn out to be undecidable. In particular, it can be shown [139, 27] that there does not 
even exist an algorithm that would decide for given Q and R whether (Q | R) = {1}. 
From this result it follows directly that it is also undecidable whether two given 
finitely presented groups are isomorphic, or whether two given words represent the 
same element of a given finitely presented group. 

Most problems can therefore only be solved heuristically or by special purpose 
algorithms applicable only to finitely presented groups satisfying certain additional 
properties, or by semi-decision procedures (methods that terminate if the answer is 
“ves” but run forever if the answer is “no”). Two methods are of particular impor- 
tance. The first is called Knuth-Bendix completion [108, 14]. This is a procedure 
that interprets the elements of R as simplification rules and constructs an enlarged 
set R’ D R with the following two properties: (1) every word @ € (QUQ7!)* can 
be brought by finitely many applications of the simplification rules in R’ to a unique 
word @ that cannot be simplified any further, and (2) two words @1, @) € (QU Oty? 
represent the same element of (Q.| R) if and only if @; = @). The Knuth-Bendix pro- 
cedure terminates if and only if there exists a finite set R’ with these properties. Com- 
putationally, the Knuth-Bendix procedure works similarly as Buchberger’s algorithm 
for computing the Grébner basis of a polynomial ideal, with the set R’ playing the 
role for the normal subgroup (R)- = (R’) C F that Grébner bases play for poly- 
nomial ideals. In particular, once a finite R’ is available, many problems that are in 
general undecidable admit simple algorithmic solutions. 

The second important method is the Todd-Coxeter algorithm [177, 167]. This 
method takes as input a finitely presented group G = (Q | R) and a finite subset 
B CG, and it produces as output a representation of the set G/H of left cosets of 
H in G. More precisely, when n = |G/H| is finite, the algorithm produces as output 
a matrix M € {1,... nyrx2|Ql in which each number 1,...,” represents one of the 
cosets and the matrix entry mj;,; is the coset obtained by multiplying the coset i by 
the jth element of QUQ7~!. If G/H is infinite, the algorithm does not terminate. By 
convention, the index | is typically assigned to the coset H = H - 1. The matrix M is 
called a coset table for G and H. The coset table can also be read as the description 
of a group homomorphism @: G > S, with @(H) = {g € @(G): g-1 = 1}. For 
improvements and applications of the Todd-Coxeter algorithm, see [167, 87] and the 
references given there. 


Example 15.3.12 (Todd-Coxeter Algorithm) Let 
G=(a, b, cla’, b, c*, (ab)’, (ac)*, (bc)*) 


and let H be the subgroup generated by a and bc in G. The Todd-Coxeter algorithm 
finds that there are eight cosets of H in G. A coset table is 
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aa! bb! ¢ c! 
Zii 1 2 3 4 2 
2|/3 4 3 1] 1 5 
3|}4 2 1 2 6 7 
4\|2 3 4 4 5 |] 
S|7 6 7 8 2 4 
615 7 6 6 8 3 
7\6 5 8 5 3 8 
8|8 8 5 7 7 6 


This means, for example, that multiplying the coset with index 5 by the element b™! 
from the right yields the coset with index 8. The coset with index 1 is H itself. Note 
that for the termination of the method it does not matter that G is an infinite group. 
The homomorphism : G — Sg sends a to (2,3,4)(5,7,6), b to (1,2,3)(5,7,8), 
and c to (1,4,5,2)(3,6,8,7). Note that @(a) and $(bc) = (1,2,3)(5,7,8) © 
(1,4,5,2)(3,6,8,7) = (2,6,8)(3,4,5) both have | as fixed point. 


Example 15.3.13 (Coxeter Groups) Coxeter groups [67, 26] are finitely presented 
groups (Q.| R) where R is of a very special form: All elements of R must be of the 
form (ab)‘, where a,b € Q and k is a positive integer, with k = 1 if a= b. Due to 
this special structure, many problems are decidable for Coxeter groups even though 
they are undecidable for arbitrary finitely presented groups. In particular, there is 
a way to decide whether two words represent the same element of a given Coxeter 
group, and it is possible to determine whether a Coxeter group is finite or not (and 
to determine its size if it is finite). 


15.3.5 Software 


Computer algebra packages for combinatorial species have already been developed 
long ago [22, 195, 70]. More recent implementations include the Maple package 
combstruct as well as packages for MuPad [85], Aldor [84], and Sage [152]. There is 
also a species library for the functional programming language Haskell [189]. These 
packages contain implementations of counting algorithms, enumeration procedures, 
and random samplers. 

Andrews, Paule, and Riese have implemented their algorithms in form of a Math- 
ematica package [10]. The algorithm of Xin is available in a Maple package [188]. 
For discrete geometry there are special purpose software systems such as poly- 
make [77] and LattE [123], which contain implementations of Barvinok’s algorithm 
as well as a number of other features not discussed here. 

Most general purpose computer algebra systems provide some functionality for 
computing with groups. Typically these packages include implementations of the 
classical algorithms for computational group theory. Some offer useful tools for ex- 
ploring and visualizing the structure of a given group, e.g., the subgroup lattice of a 
given permutation group. A description of the GroupTheory package for Maple 17 
can be found in [149]. 
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More advanced operations and more efficient implementations are provided by 
the special purpose system GAP [76]. At the time of writing, this system defines the 
state of the art in software for computational group theory. The functionality of GAP 
is also available in Sage [174]. 


15.4 Symbolic summation 


Symbolic summation refers to algorithms for simplifying expressions involving the 
summation sign. In view of the ubiquity of sums in all parts of combinatorics, such 
algorithms are obviously of considerable importance. 


15.4.1 Classical algorithms 
15.4.1.1 Hypergeometric terms 


Classical summation algorithms deal with sums over hypergeometric terms. Let C be 
a computable field and let A = C(k)[S;] be the Ore algebra of recurrence operators 
(see Section 15.2.4). A nonzero element / of a function space M on which A acts is 
called hypergeometric (over C(k)) if it is the solution of some operator a € A of the 
form a = S,—r, with r € C(k) \ {0}. 

In other words, 4 is hypergeometric if the shift quotient (S; -/) /h can be identi- 
fied with a rational function r € C(k) \ {0}. In yet other words, h is hypergeometric if 
it can be written as a product A = []‘<j r(é) for some rational function r € C(k) \ {0}. 


For example, 4 = 5k 1S a hypergeometric term because we have 


k! 
(Qk+1) 


spe 241+) 
 Q(R+ 1) + I)26! FER (24 1)22* 22K +3) 


Hypergeometric terms together with multiplication form a group. The sum /; + 
hy of two hypergeometric terms /1,h2 is hypergeometric if and only if h2/h, can 
be identified with a rational function. In this case, we call h; and h2 similar. By 
definition, every hypergeometric term h is similar to its shift S;, +h. 

For the case of more variables, let A = C(k1,...,km)[Sk,,---,Sk,]. A nonzero 
element / from a function space M on which A acts is called hypergeometric if for 
every i= 1,...,m there exists some operator a € A of the form a = S;, — r; for some 
ri © C(k1,.--,km) \ {0} such that a-h = 0. Again, another way of saying this is that all 
the shift quotients (S;,-h)/h (i = 1,...,m) must agree with some rational functions. 
For example, h = eS is hypergeometric over Q(n,k), because we have 


Sch (x01) _ n-k Sek oC). n+l 
i ee a, eS 
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A hypergeometric term in n,k is called proper if it can be written in the form 


M 
px'yk T(ajn + bik +c) 
i=l 
where p € K[n,k] is a polynomial, x,y are constants, M is a fixed integer, the a;,b;, e; 
are fixed integers, the c; are constants, and the shift acts as expected, e.g., 
1 
(2n — 3k)(2n — 3k + 1)(2n —3k +2) 


S,-T(2n—3k +3) = T'(2n—3k +3). 


For example, h = @) is proper hypergeometric because we can write it as h = 


ite: which is of the required form. On the other hand, it can be shown 
1 


that the hypergeometric term h = — +e is not proper [186]. 

The notion of hypergeometric can be generalized to the g-case. Here, we let C 
be a field of characteristic zero, g € C \ {0} some distinguished element, typically 
assumed not to be a root of unity, and we consider the Ore algebra A = C(x)[0] 
defined by o(x) = gx (compare Examples 15.2.12.(2) and 15.2.13.(2)). A nonzero 
element of some function space M on which A acts is called g-hypergeometric if it 
is the solution of some operator a € A of the form a = 0 —r for some r € C(x) \ {O}. 


For example, g@) is g-hypergeometric because it is annihilated by the operator 0 — x. 
Multivariate (proper) g-hypergeometric terms are defined analogously, see [186, 142] 
for details. 

In most of the symbolic summation literature, hypergeometric terms are regarded 
as formal objects living in some extension field of the field of rational functions (see 
also Section 15.4.2 below for a more precise description). In combinatorial applica- 
tions, it is usually necessary to interpret hypergeometric terms as sequences. Users of 
symbolic summation algorithms should be aware that this subtle difference can lead 
to misunderstandings about what a certain computation has or has not proved. See 
Chapter 5 of [102] for a discussion of hypergeometric summation algorithms that 
emphasize these issues. 


15.4.1.2 Gosper’s algorithm 


An expression of the form )°7_, / is called indefinite sum if the upper summation 
bound is a variable, and this variable does not appear in the summand expression hx. 
Finding a closed form expression for an indefinite sum is equivalent to finding a 
closed form solution g, of the so-called telescoping equation g7+1 — gn = hy. For if 
8y is such a solution, then we have Y°?_ hk = 8n+1 — 80- 

Gosper’s algorithm [81, 82, 141, 145, 109, 102] solves the indefinite summation 
problem for the case when the summand expression /,% is a hypergeometric term. 
That is, given a hypergeometric term h, it decides whether there exists another hy- 
pergeometric term g such that o(g) — g = h, and if yes, it constructs such a term. 
A hypergeometric solution g of the telescoping equation must necessarily have the 
form g = ch for some rational function c. Gosper’s algorithm can therefore also be 
considered as an algorithm for deciding whether there exists a rational solution c to 
an equation of the form ro(c) —c = 1, where r is a known rational function. 
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Example 15.4.1 (Indefinite Hypergeometric Summation) 
1. Gosper’s algorithm finds that Y7_ md Cc ) can be simplified to (1+2n)q (ees 


2. It also proves that Yy_, i does not have a hypergeometric closed form. 


15.4.1.3 Zeilberger’s algorithm 


A sum that is not indefinite is called definite. In particular, sums whose upper sum- 
mation bound is not a variable (as, e.g., in Ve. (i). and sums whose upper sum- 
mation bound is a variable that also appears in the summand expression (as, e.g., in 
Yi-o (7) are definite. 

Zeilberger’s algorithm [190, 192, 145, 109, 102] solves the following problem: 
If hyx is bivariate hypergeometric over C(n,k), it searches for a nonzero operator 
P €C(n)[S;] and a rational function Q € C(n,k) such that (P— (S;— 1)Q) - An, = 0. 
Such a pair (P,Q) is called a creative telescoping relation. P is called a telescoper 
for h,,, and Q a certificate for P and h, x. 

A creative telescoping relation for h, , gives rise to a recurrence equation for the 
definite sum F (n) = Y7_o/nxe. Writing (P — (S;— 1)Q) - hn, = 0 in the form 


Po(n)Ank + Pi(a)hnsik tes? + Pr(n)hngre = (nk + Dang — a(n, kK hin 


and summing this equation over k = 0,...,n, we get 


po(n)F (n) + pi(n)(F(n+ 1) —hngingt) +2 + p(n) i( n+r)— Emirs) 


= q(n,n + I)Ann+1 os q(n,0)hn.o, 
which can be rearranged to 
po(n)F (n) + pi(n)F (n+ 1) +--+ pr(n)F (nr) = ths(n) 


for some explicit right-hand side. In summation problems arising in applications, the 
right-hand side usually turns out to be zero. 

Not every hypergeometric term admits a creative telescoping relation. Zeil- 
berger’s algorithm finds such a relation if and only if one exists. A sufficient condition 
for the existence of a creative telescoping relation is that the summand be a proper 
hypergeometric term. This covers most situations arising in applications. It is how- 
ever not necessary. A necessary and sufficient condition was given by Abramov [3]. 


Example 15.4.2 (Definite Hypergeometric Summation) 


L. For hag = a Zeilberger’s algorithm finds the telescoper P = (n+1)Sn— 


(4n +2) and the certificate Q = ee) For the sum F (n) = Ye. Cy = 


Yio (ae , it follows that (n+ 1)F(n+1) = (4n+2)F(n). Together with the 
tate caer : Al dn 
initial value F (0) = 1, this implies F (n) = TT a — eer 


n 
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2. [143, Section 4.3] For hy, = (-1)¥ (2) Cy, Zeilberger’s algorithm finds the 


telescoper P = (4n +6)S> + (15n +21)S, +(9n+9) and the certificate Q = 
(n—3k) (n—3k+1) (n—3k+2) (2n+3) 
(n+2)(n—k+1)(n—k+2) 
currence P- F(n) = 0. Among all telescopers for a given hypergeometric 
term, Zeilberger’s algorithm always finds one that has smallest possible order. 
However, the sum may still satisfy a lower order recurrence. In the present 
example, we have F(n) = (—3)", which satisfies the first order recurrence 
F(n+1)+3F(n) =0. Such examples are rare. 


. For the sum F(n) = Vi_o hn, we obtain the re- 


3. Not all definite hypergeometric sums admit a hypergeometric closed form. As 


2 
an example, the term hak = (")? Ce has the telescoper P — (n + 1) _— (2n + 


2 
3)(17n2 + 51n +39)Sy + (n+2)382. For the sum F(n) =Y2_o (")" ("£*) 
have P- F(n) = 0, but there is no first order operator that annihilates F (n), 
i.e, F(n) is not hypergeometric (see Example 15.4.3.(2) below for a proof). 
This sum plays a role in Apéry’s proof [180] of the irrationality of € (3). 


4. For hyay = GA, Zeilberger’s algorithm finds the telescoper P = S, — 4 and the 


certificate Ome Oey For the sum F(n) =Yi_ @) we therefore get the 
inhomogeneous recurrence F (n+ 1) —4F (n) = — ay eae 


15.4.1.4 Petkovsek’s algorithm 


In general, Zeilberger’s algorithm does not deliver a closed form for a given definite 
hypergeometric sum, but only a recurrence that has the sum among its solutions. In 
the lucky case where this recurrence happens to be homogeneous and of first order, 
the recurrence can easily be rephrased as a hypergeometric closed form for the sum. 
In the general case, this is not so easy. 

PetkovSek’s algorithm [144, 145, 109, 102] solves this problem: Given a recur- 
rence operator P € C(n) [Sp], it decides whether P has hypergeometric solutions, i.e., 
whether there exist hypergeometric terms h over C(n) with P-h = 0. In the affirma- 
tive case, it finds them all. 

By definition / is hypergeometric over C(n) if there exists an operator H = uS,, + 
v € C(n)[S,] \ {0} with H -h =0. If P is any operator, then A is a solution of P if 
and only if H is a right factor of P, i.e., we can write P = UH for some operator 
U €C(n)[S,]. We can therefore say that PetkovSek’s algorithm solves the problem of 
finding all the first order right factors of a given operator P. 

Factoring Ore polynomials is computationally much harder than factoring com- 
mutative polynomials. The runtime of PetkovSek’s algorithm is exponential in the 
number of irreducible factors of the leading and the trailing coefficient of the input 
operator. Better algorithms are known (see [183]), but they also require exponential 
time. 
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Example 15.4.3 (Hypergeometric Solutions) 


1. Suppose that for a certain definite sum F (n), Zeilberger’s algorithm produced 
the recurrence 


8(2n+5)(6n+ 11)F(n+3) —6(24n? + 102n + 101)F (n+2) 
+3(18n? +73n+64)F(n+1)—(n+1)(6n+17)F(n) =0. 


PetkovSek’s algorithm finds that this recurrence has two hypergeometric solu- 
tions: ea) and rae Being of third order, the recurrence has another solution, 
linearly independent of those, which is however not hypergeometric. 


The sum F(n) admits a closed form in terms of hypergeometric terms if and 
only if F(n) = a.(°) + B qr. To determine the constants a and B, if they exist, 
consider the system of linear equations obtained from this ansatz by setting 
n=1andn=2. If we have F(1) = 3 F(2) = a F(3) = 388 then we 
find & = 3,8 = —1 and thus F(n) = 3(") — a for all n. If instead we had 
F(1)=4, F(2)= ip F (3) = , there would be no solution 0, B, and therefore 
no hypergeometric closed form of F (n). 


2. We have seen in Example 15.4.2.(3) that the sum F (n) = Vp (") (ens sat- 
isfies a certain second order recurrence. According to Petkovsek’s algorithm, 
there is no hypergeometric solution of this recurrence. In particular, the solu- 
tion F (n) cannot be hypergeometric, viz. it does not satisfy a first order recur- 
rence. Consequently, the recurrence found by Zeilberger’s algorithm for F (n) 


has minimal order in this example. 


15.4.2 [[£-Theory 


The classical algorithms for hypergeometric summation as outlined in the previous 
section have been generalized in various directions. The two most striking lines of 
development have led to sophisticated algorithms based on difference fields, as dis- 
cussed in the present section, and based on operator ideals, as discussed in the fol- 
lowing section. 


15.4.2.1 II£-Expressions and difference fields 


A difference field is a field K with a distinguished field automorphism o: K — K. 
The elements of a difference field K are thought of as (formalizations of) sequences, 
and the automorphism o is meant to act on elements of K like the shift operator acts 
on the corresponding sequences. Elements c € K with o(c) = c are called constants. 
The set of all constants of K forms a subfield of K. 

IIx-fields are special difference fields introduced in 1981 by Karr [93, 94]. They 
are designed to formalize expressions that can be formed (with certain restrictions) 
from numbers, variables, arithmetic operations, summation sign and product sign 
according to the usual syntactic rules. [12-fields are constructed recursively from 
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simpler I1X-fields by “adjoining a sum or a product.” The construction starts with an 
arbitrary constant field. 

More precisely, if K is a I1X-field with constant field C C K and automorphism 
o: K > K, then the field E = K(t) of rational functions in t with coefficients in K is 
turned into a difference field extending o to E via o(t) = at+ for some a € K\ {0} 
and B € K. Observe that when a = | then the new element t represents an indefinite 
sum over an element of K, and when B = 0 then ¢ represents an indefinite product 
over an element of K. If @ and B are such that all elements of E left fixed by o are 
in C, then E is again a IT2-field. 


Example 15.4.4 (II-fields) Expressions such as 


Rod 

“KY + Via 7 
k 

fm Ok +k 
can easily be modelled as elements of IYx-fields. In order to construct a suitable 
field, start with the constant field C = Q. Going through the expression from inside to 
outside, adjoin a new transcendental element for every sum or product encountered. 
Note that n = ¥°"_, 1 can be viewed as a shorthand notation for a sum, and 2" and 
n! are obvious shorthand notations for products. Thus we obtain the Y1X-field K = 
Q(t ,t2,t3,t4,t5) with o defined via 


=ft+1 “than” 


-_ 2to “ to ~ Qn ” 


1+(t)+1)t3+(t)+1)2ty 
+1) (1 +142r) 


Observe that there is some freedom in the arrangement of the field extension, but it is 
not completely arbitrary. For example, we may exchange t and tz, but not t, and 13, 
because O(t3) involves ty. 


It is not obvious (but true) that all the constants of the field K constructed in the 
example above belong to Q, as is required for K to be a IIX-field. As an example 
for a problematic construction, suppose we wanted to construct a IIX-field for the 
expression (—1)”. A first attempt is to take Q(t) with o(t) = —f, but then o(t?) = 
o(t)* = (—t)* =f, sot? € Q(t) \Q would be a new constant. Such new constants are 
forbidden in a IIX-field. A second idea might be to factor out the algebraic relation 
((—1)”)? —1 =0, taking QJt]/(t? — 1) instead of Q(t), but since t? — 1 = (t—1)(t+1) 
is not irreducible, this is not a field. In fact, Karr’s original theory excludes sums 
involving the alternating sign. However, more recent versions of the theory developed 
by Schneider [155] also cover alternating signs. 

For an expression involving sums and products to admit a natural translation 
into a IIX-field, it is necessary that the nesting is indefinite. This means that every 
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subexpression must contain at most one free variable. For example, all the boxed 
subexpressions of 


= 
— 


>I 
Me 
| 


e 


~. 


k 1 


II 
ip 


U 


contain only one free variable (and possibly several other variables that all are bound 
by a summation sign), so this expression is admissible. On the other hand, the boxed 
subexpression in 


a 1 
d n+k+1 


contains two free variables n,k, so it is not admissible. We call expressions that are 
admissible in this sense IIx-expressions. There is an algorithm that takes a ITx- 
expression as input and produces a IIX-field containing a corresponding element. 

Some care must be applied when ITX-expressions are interpreted as sequences. 
For example, the expression n(n — 1)(n — 2) []f_, (k* — 7k + 12) represents the zero 
sequence but it may correspond to a nonzero element of a IIX-field. If such an ex- 
pression appears in the denominator of another expression, this may lead to results 
that are only correct in a certain algebraic sense but give no meaningful information 
about sequences. Fortunately, such issues rarely arise in practice, and for a large class 
of ITX-expressions it can even be proven that they won’t arise [161]. 


15.4.2.2 Indefinite summation 


Karr’s algorithm solves the indefinite summation problem for I2-fields: Given a 
IIx-field K and an element f € K, it decides whether there exists an element g € K 
such that o(g) — g = f, and if so, it finds such a g. 


Example 15.4.5 (In-Field Summation) Let H,, = Yi_, i Then Karr’s algorithm 
can find the identities 


n 
\) A = (n+ 1) A, —1, y= 2n — (2n+1)Hy + (n+1)H? 


> 
Il 
fae 


in the sense that given the left-hand side, it can calculate the right-hand side. The fact 
that the algorithm does not find any simplification for the sum V7_ | HR is a proof that 
this sum cannot be written as a rational function in n and Hy. 


Karr’s algorithm includes Gosper’s algorithm as a special case. Indeed, if h is 
hypergeometric over Q(n), then Q(n)(h) is a T1=-field, and if there exists another 
hypergeometric term g with o(g) — g =A, then this hypergeometric term must belong 
to Q(n)(h), so Karr’s algorithm will find it. 

Karr’s algorithm for simplifying I1X-expressions can be viewed as a difference 
analogue of the Risch algorithm [150, 46] for deciding whether the integral of a 
given Liouvillian function (i.e. an expression composed of numbers, a variable x, 
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and the functions exp and log) is again Liouvillian. In this analogy, log-expressions 
correspond to sums and exp-expressions correspond to products. 

There is an extension of Karr’s algorithm due to Schneider [160] that finds sim- 
plifications of a given sum that do not necessarily belong to the IIX-field in which 
the summand lives. Define the depth of an element of a IIX-field K = C(f1,.-..,tn) 
recursively by setting the depth of the elements of C to zero, the depth of other el- 
ements to the maximal depth of all the generators ¢; appearing in it, and the depth 
of a generator 1; satisfying o(t;) = at; + B as 1 more than the maximum depth of a 
and B. 

The depth of an element of K reflects the nesting depth of the corresponding 
TIx-expression. For example, if K = Q(t1,t2,t3,t4,ts) is the I1=-field from Exam- 
ple 15.4.4, then ¢; and t) have depth 1, #3 and ty have depth 2, and ts has depth 3. 
Schneider’s extension of Karr’s algorithm takes as input a I12-field K and an ele- 
ment f € K, and it constructs as output an extension field E of K and an element 
g © E such that o(g) — g = f and g has minimal possible depth. 


Example 15.4.6 (Nesting Depth Reduction) 


1. We have seen in the previous example that the sum "_, H} cannot be ex- 
P Ip k=1"k 
pressed as a rational function in n and Hy. However, Schneider’s algorithm 
finds the representation 


Observe that the sum in the last term on the right does not appear as a subex- 

pression of the summand. Note also that the right-hand side is simpler than 

the left-hand side in the sense that Y7_, He = eS (yo a has depth 3 
j=l 

while the right-hand side has only depth 2. 


2. Schneider’s algorithm proves that Y7_, Hi does not admit any representation 
in terms of single sums. 


3. Also the five-fold sum 


: 1 
m di 
ez : 
k Wel m2 
n Py i] 
2X KE 


does not admit a representation in terms of single sums. But it does have 


the following representation in terms of double sums, where we write HY” a 
Ye a for the harmonic numbers of order r: 
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(4) n (2))2 
1/1 2)\3 2 74) 1 ey 2 _ Yr Ae Ae (Ay) 
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ala k2 3 k k 
k=1 k=1 k=1 
n H? n H; ‘ n H? n H) H; n HR 
2). 4 + » 2 '°"n 2 2 L 2 2) 3 
k=1 =1 k =1 k k=1 k k=1 k 


15.4.2.3 Creative telescoping 


When two elements /|, f2 of a IZ-field K are not summable in K, i.e., there is no 
g €K with o(g)—g = f| and also no g € K with o(g) — g = fy, it may still be the 
case that there exist constants c,,c2 in K such that the linear combination c, f, + c2f2 
is summable. Given fo,..., f; € K, the set 


V:= {(8,€0)---,¢r) EK XC"! : o(g)—g =cofot-+erfr} 


is a vector space over the constant field C of K. Given K and fo,...,f-, Karr’s al- 
gorithm can compute a basis for this vector space. This is known as parameterized 
telescoping. Note that it contains the usual telescoping problem for r = 0. 

The parameterized telescoping problem arises as a subproblem of Karr’s algo- 
rithm for solving the telescoping equation. Schneider [155] observed that it can be 
used to do creative telescoping in IIX-fields, thus generalizing Zeilberger’s algorithm. 

The procedure applies to any expression in two variables n,k which is a IIX- 
expression with respect to k when n is regarded as a constant, and which maintains 
this property when n is replaced by n +i for any fixed i € N. If f(n,k) denotes such 
an expression, we choose some r > 0 and set fo,..., f; to the elements correspond- 
ing to f(n,k),...,f(n+7,k) in a suitable I1X-field K. Then we use parameterized 
telescoping to find, if possible, a nontrivial linear combination co fo + ---+c;f; with 
constants c; (that may well involve n, but not k) that is equal to o(g) — g for some 
geek. 

If no such relation is found, increase r and try again. The procedure will find a 
creative telescoping relation after finitely many steps if and only if one exists. If there 
is no such relation, it will run forever. There are no results guaranteeing the existence 
of creative telescoping relations for (a reasonably large subclass of) the expressions 
to which the method applies, but the experience is that they do exist for all sums 
arising in combinatorics. 


Example 15.4.7 (Apéry’s second sum) [/80, 158] Apéry’s proof of the irrational- 
ity of € (3) involves the sum 


ro E(t) (Cr!) Eee Baraen) 


Write f(n,k) for the summand expression. When n is fixed, this expression is a T1X- 
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expression with respect to the variable k. Also the two shifted expressions 
(nt+k+1) (n *(n +k’ 
1,k) = ———_—_ 
TT NO espa? Vi k 
fl 1 k (n—i+1)(-1) 
«(Dat TEE ee “3 . n a) 
11 (Al) 37 oe () 
(n+k+2)?(n+k+1)* (n ae 
2,k) = 
a aay OE OS) ae FA cee ap! 
“i 1 y erento 


(n+2)) S28 4i+2DH+i- DO (PF 


U 


are II%-expressions with respect to k when n is regarded as a constant. We can con- 
struct a I1X-field K with constant field C = Q(n,c) (where c is used for representing 
the constant sum Y"_, a); such that K contains elements fo, f, f2 corresponding to 
f(n,k), f(n+1,k), f(n+2,k), respectively. Parameterized telescoping applied to 
fo, fi, f2 € K gives rise to a creative telescoping relation for f(n,k), which in turn 
gives rise to the recurrence 


(n+1)3F(n)— (2n+3)(17n? +51n+39)F(n+1)+(n+2)F(n+2) =0 


for the original definite sum. This is the same recurrence as in Example 15.4.2.(3). 


15.4.2.4 D’Alembertian solutions 


Given a linear recurrence equation as produced, for example, by creative telescoping, 
we may wish to determine whether it admits solutions that can be represented by 
suitable elements of I1X-fields. In general, this is not the case, but for sums arising in 
applications, it often is. 

Consider an operator P € C(n)[S,] with rational function coefficients. The first 
order right-hand factors of P can be found using PetkovSek’s algorithm (see Sec- 
tion 15.4.1). They correspond exactly to the hypergeometric solutions of P. Suppose 
(S, —u) for some u € C(n) is a right-hand factor of P, and write P = U(S, — u) for 
some U € C(n)[S,]. One solution of P is then given by the product hy = []{_) u(k — 
1), which evidently can be viewed as an element of a ITZ-field. Suppose also U has 
some first order right-hand factor (S,, — v) with v € C(n), so that U = V(S, — v) for 
some V € C(n)[S,], and P = V(S, —v)(Sp —u). Then f, = []f_, v(k — 1) is a so- 
lution of U, and gy, := (Te u(k — 1)) Vier We nae is a solution of P because 
(Sy —U) + 8n = fn. Also this solution can evidently be viewed as an element of a ITZ- 
field. Next, any hypergeometric solution of V translates into a single sum solution 
of U, which in turn translates into a double sum solution of P, and so on. 

In the best case, this procedure leads to r linearly independent solutions 
of P as nested sums and products. Such solutions are called d’Alembertian [6]. 
D’ Alembertian solutions can be computed more generally for linear recurrence equa- 
tions whose coefficients belong to an arbitrary ITX-field instead of just C(n), and for 
inhomogeneous recurrences [4]. 
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15.4.2.5 Nested definite sums 


IIx-expressions are composed of nested sums and products, but all sums and prod- 
ucts arising in a II2-expression are indefinite in the sense that the upper bound of 
every summation or product sign is a variable that does not occur in the correspond- 
ing summand or factor expression. In the context of X-theory, these expressions 
play the role of “closed forms.” A typical application scenario is that a combinatorial 
problem leads to a definite sum whose summand is a ITZ-expression, and we wish to 
know whether the definite sum can be simplified to a 1Z-expression. Definite single 
sums are simplified by the following steps: 


1. Using creative telescoping, compute a linear recurrence for the sum. 

2. Compute the d’ Alembertian solutions of this recurrence. 

3. If possible, write the original sum as a linear combination of these solutions. 
4. Compute an equivalent ITZ-expression of minimal depth. 


Nested definite sums are simplified by applying these steps repeatedly, starting from 
the innermost definite sum and working outwards: 


y y TIL-expression w.r.t. 7 ba y IIX-expression w.r.t. 7 


with n,i as parameters <4 | with n as parameter 


i=0| j=0 


See the survey articles [156, 163, 162] for details and examples. 


15.4.3. The holonomic systems approach 


There are quantities that do not admit a closed form representation in terms of sums 
and products or other standard objects. Some of these can still be expressed as solu- 
tions of certain equations. In a sense, this is analogous to the choices for representing 
numbers (cf. Sect. 15.2.1): The numbers that can be expressed in terms of radicals 
form a proper subset of the algebraic numbers, and those in turn form a proper subset 
of all numbers. All algebraic numbers, even those that can not be written as radicals, 
can be described by the polynomial equation of which they are a root. 

The situation is the same for power series. A formal power series a € K|[[x]] 
is called algebraic if there exists a nonzero polynomial p € K[x,y] such that 
p(x,a(x)) =0. The polynomial p together with a finite number of terms of a uniquely 
determine the infinite object a by a finite amount of data. We can therefore do rigor- 
ous computations in the broad class of algebraic power series. An even broader class, 
discussed in the present section, is the class of D-finite functions. In the simplest case, 
these are power series a € K[x]] that satisfy linear differential equations with polyno- 
mial coefficients, or sequences (a,)*°_p that satisfy a linear recurrence equation with 
polynomial coefficients. Again, the defining equation together with a suitable num- 
ber of initial values uniquely determines such an infinite object by a finite amount of 
data. In this section we discuss algorithms operating on such representations. 
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15.4.3.1 D-finite and holonomic functions 


In its most general form, the notion of D-finiteness extends to objects in several 
variables that live in some vector space M on which an Ore algebra A = K[01,..., | 
acts. We then say that an element f € M is called D-finite (with respect to the action 
of A on M) if for all i € {1,...,n} we have ann(f) NK [d;] 4 {0}. 

For sequences and power series in a single variable, to be D-finite just means 
that there is one nonzero operator (in K(n)[S,] or in K(x)[D,], respectively) which 
annihilates it. For objects in several variables, say some object a(x1,...,%u,711,---,7v) 
depending on u “continuous” variables x;,...,x, and v “discrete” variables n1,...,7y, 
the definition says that it is D-finite if and only if it satisfies for eachi=1,...,ua 
linear differential equation with polynomial coefficients involving no shifts and only 
derivatives with respect to x;, and for each j = 1,...,v a linear recurrence equation 
with polynomial coefficients involving no derivatives and only shifts with respect 
to n;. There is no restriction on the variables that may show up in the polynomial 
coefficients. 

Regardless of whether or not it is the annihilator of a function, an ideal a is called 
D-finite if it has the property aN K[0,] 4 {0} for all i. In typical applications, a basis 
for the whole annihilator of a function f may not be easy to obtain. To cover also 
these situations, we say that an ideal a is a D-finite description for f if a is D-finite 
and ann(f) C a. In most cases, multivariate D-finite objects can be specified by a 
D-finite description plus a finite number of initial values. However, the situation is 
not always as straightforward as in the univariate case. 


Example 15.4.8 (Initial Values in the Multivariate Case) 
1. The sequence f(n,k) = 3" + (3nk + 1)2* is D-finite. A D-finite description is 
given by the ideal a = (a,,a2) C Q(n,k)[Sn, Sx] where 
ay = (6kn — 3k +.2)S2 — (24kn — 6k +8)S, + (18kn + 9k +6), 
ay = (3kn +. 6n + 1)S? — (9kn — 24n — 3)S, + (6kn + 18n+2). 
Note that aN Q(n,k)[Sn] = (a1) 4 {0} and aN Q(n,k) [Sx] = (a2) F {0}. 


Using these two recurrences, every term f(n,k) forn,k © N can be computed 
recursively from the four initial values f (0,0), f(1,0), f(0, 1), f(, 1). 


2. For doing computations it is not necessary (and not advisable!) to represent 
the ideal a by a basis ay,...,a, such that aM K[0;| = (a;). The definition only 
requires that such elements a; exist and not that we explicitly know them. For 
example, the ideal a from part I can also be generated by the operators a, and 


a3 = (6kn — 3k +2)S; + (3kn + 6n + 1)S, — (15kn + 18 — 3k +5). 


Note that with this basis all the terms f (n,k) can be computed recursively from 
only two initial values f (0,0), f(1,0). In general, it is preferable to work with 
bases for which the number of terms needed to identify a particular solution is 
as small as possible. The number of initial values is minimal when the basis is 
a Grobner basis. 
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We have seen in Example 15.2.14 for the univariate case that when a recurrence 
has integer singularities, we may need to specify a finite number of additional initial 
values. Singularities in the multivariate case are integer roots of multivariate poly- 
nomials. This is more troublesome because (a) there may be infinitely many, and 
(b) there is no algorithmic way to find them all [132]. 

If f is D-finite with respect to the action of the algebra K(x1,...,%u)[Dx,,---,Dx,] 
of differential operators, then we also say that f is holonomic. For a multivariate se- 
quence, or more generally, for an object f = f(x1,...,%u,71,---,”y) depending on 
u continuous variables x1,...,x, and v discrete variables n,...,n, on which the al- 
gebra K(x1,...,%u,M1,...,My)[Dx,,---,Dx,,Sn,;-++,Sn,] acts, we say that f is holo- 
nomic if its generating function 


3 oie y f BijeesRani cee ye ty ey 


n,=0 ny=0 


is holonomic (or D-finite) with respect to the action of the algebra of differential 
operators K(x1,...,Xu,Z1,---,2v)[Dx,,---,Dx,,Dz,,--+,Dz,]- 

This distinction between D-finite and holonomic is subtle, and it plays hardly any 
role in practice. Typically, an object is either D-finite and holonomic or it is neither 
holonomic nor D-finite. However, none of the two notions implies the other. For 
example, the sequence 1 /(n? +k?) is D-finite but not holonomic, and the Kronecker- 
delta function On.k viewed as a bivariate sequence is holonomic but not D-finite. 


15.4.3.2 Closure properties 


In applications, a function f will usually not be given in terms of a D-finite descrip- 
tion but by some sort of expression. It is therefore important to recognize whether a 
given expression in fact represents a D-finite object, and if so, to construct a D-finite 
description for it. 

This is done using the concept of closure properties. These are theorems that state 
that D-finiteness is preserved under various operations. By means of these theorems, 
it can often be seen “by inspection” that some complicated expression is D-finite, by 
looking at the innermost subexpressions first, which often are D-finite by definition, 
and then observing that the expression is composed of these subexpressions only 
through operations that preserve D-finiteness. 

Of course, if the D-finiteness of a quantity cannot be established through closure 
properties, it may still be D-finite for other reasons. In fact, it can be very difficult to 
prove that some object is not D-finite. See [79, 74] and the references given there for 
some results in this direction. 

The most important closure properties are the following. See [191, 173, 63, 110, 
102] for proofs and further details. 


1. Let A be an Ore algebra acting on a function space M. If f,g € M are D-finite, 
then also f+ is D-finite. If f € M is D-finite and L € A, then L- f is D-finite. 
If M is also aring, and f,g € M are D-finite, then also fg is D-finite. 
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2. Let A = K(x)[D,] act on M = K|[x]], where K is a field of characteristic zero. 
Let f = LP_o fnx” € M be D-finite. Then also ff := Yr) Int yn is D-finite 
with respect to A, and the coefficient sequence (f;,);°_9 in KN is D-finite with 
respect to K(n)(S,,]. Furthermore, if g € K|[x]] is algebraic with g(0) = 0, then 
also fog € K|[x]] is D-finite with respect to A. 


3. Let A = K(n)[S,] act on M = KN, where K is a field of characteristic zero. Let 
(fr)e_9 © M be D-finite. Then also Y f := (LiLo fi);°_p is D-finite with respect 
to A, and the generating function f => 9 frx" € K|[x]] is D-finite with respect 
to K(x)[D,]. Furthermore, if u,v € N are fixed, then (fin+));°_) € M is D-finite 
with respect to A. 


These closure properties are constructive. This means that if D-finite descriptions 
a and b for two D-finite objects f and g are known, then we can compute a D-finite 
description of f + g, fg and the other quantities. In the case of a single variable, if f 
and g are annihilated by operators a and b, respectively, then f + g is annihilated by 
the least common left multiple Iclm(a,b). The corresponding operation for multi- 
plication is called the symmetric product: if c is the symmetric product of a and b, 
then c- (fg) =0. 

Closure properties are useful for recognizing expressions as D-finite objects, but 
can sometimes also be used to construct algorithmic proofs of identities among D- 
finite objects, as illustrated in the following example. 


Example 15.4.9 (Proving Identities) This example is borrowed from [99]. Con- 
sider the following identity for Hermite polynomials. We regard it as a (formal) power 
series with respect to t, where x and y are viewed as constant parameters. In the first 
term on the left, the expression H,(x)Hy + is regarded as a sequence in the dis- 
crete variable n, with x and y as parameters. Apply the closure properties algorithms 
as indicated by the braces in order to obtain a linear recurrence for the coefficients 
in the series expansion of the whole left-hand side as follows. 


= 1 1 At (xy —t(x? +y*)) 
Yi An(x)An(y) | 0” Aaa exp( az ) =0. 
n—0 n. 1—4t i 
Seas ees” —S— ——nr SY SS ——~’” 
rec. of rec. of rec. of alg.eq. diff-eq. alg.eq. of degree 1 
ord.2 — ord.2 ord. 1 of deg.2 of ord. 1 
e—{S>__—__“‘_ ———_ 
rec. of order 4 ae differential equation of order 1 


es A 


differential equation of order 5 differential equation of order 1 


—— 


differential equation of order 5 


~~ recurrence equation of order 4 


If Cn denotes the coefficient of t” in the series expansion of the entire left-hand side, 
we obtain that the sequence (Cn)-_9 is annihilated by the recurrence operator 


(n+4)S* — 4xy S$? —4(2n — 2x? — 2y? +5)S? — loxyS, +16(n +1). 
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Direct calculation confirms that co = c, = c2 = c3 = 0, which together with the re- 
currence implies inductively that cn = 0 for all n = 0. This proves the identity. 


15.4.3.3 Summation and integration 


The concept of creative telescoping extends to holonomic functions. It is used for 
obtaining a D-finite description for a sum or an integral given a D-finite description 
for the summand or integrand. In the summation case, suppose we know a D-finite 
description a C C(k,x1,...,Xy)[Sx,01,-.--,0,] for the summand f(k,x1,...,xy), where 
k is the summation variable and x,,...,x, are other variables, on which the 0; act in 
some way. Then a nonzero operator P € C(x1,...,X,)[01,-.-,0] free of k and S, is 
called a telescoper for f if there exists another operator Q, called a certificate for P, 
such that P—(S,—-1)Q € a. 

From an ideal p C C(xq,...,x,)[01,...,0)] of telescopers with the property 
pOC(x1,..-,x,)[0;] 4 {0} we can obtain a D-finite description for the sum 
ee f(k,x1,..-,Xy). In the simplest and most common case, when the sum has “nat- 
ural boundaries,” p itself is already a description of the sum. To have natural bound- 
aries means that lo tf et +! 0 for all operators Q. This is the case, for example, when 
the variables x),...,x, are discrete and for every specific point (x1,...,x,)) € Z” there 
are only finitely many values k such that f(k,x,...,x)) 40. In this situation, we also 
say that “f has finite support.” 

When f does not have natural boundaries, creative telescoping still gives rise to 
inhomogeneous functional equations of the form P- F = u, where F is the sum over f 
andu=(QO-f as In this case, if we know telescopers P|,..., Pn with corresponding 
certificates Q),...,Qm, and if we know (or can compute) for each i an ideal 4% of 
annihilating operators for the functions [Q;- f (‘au a = then the product ideal (q,N---M 
dm)(P1,---,Pn) is an ideal of annihilating operators for the sum F’. It will be D-finite 
provided that (P;,...,P,) and all the q; are D-finite. 

Several algorithms are available for finding telescopers for holonomic functions. 
Zeilberger’s “slow” algorithm [191, 63] uses a brute force elimination approach, 
Takayama’s algorithm [175, 176, 63] reduces the problem to a Grébner basis compu- 
tation in an Ore algebra, Chyzak’s algorithm [61] is a generalization of Zeilberger’s 
algorithm for hypergeometric summation (Section 15.4.1), and the algorithm by 
Chen, Kauers and Koutschan [57, 111] is a generalization of an idea proposed by 
Apagodu and Zeilberger [136] for the hypergeometric case. For holonomic func- 
tions, it is guaranteed that these algorithms will find enough telescopers to generate 
a D-finite ideal. 


Example 15.4.10 (Apéry’s second sum once more) Consider once more the sum 


riy= 3 (t) Gs) (ha 2p ther) 


In order to find a recurrence for this sum, process the sum from inside to outside. 
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The inner summation F\(n) =YL_, 5 is univariate and can therefore be handled 


by the closure properties algorithms discussed in the previous section. Using that 
(i+1)38; —# annihilates the summand, the result is that 


A= (n+2)? 82 — (2n+3)(n? +3n+3)S, + (n +1)? 


annihilates the sum F,(n). If we regard this sum as a bivariate object depending on n 
and k, a D-finite description is given by the ideal a, = (A,S,— 1) C Q(n,k)[Syp, Sx]- 


ee oe 
i=1 33 (") has a hypergeometric summand 


with annihilator ((i+1)(i—n)(itn+1)S;-—@, (G+n+1)8,+(i-n—-1)) C 
Q(n,i)[Sn,S;]. Given this information, a summation algorithm can find a D-finite 
description az C Q(n,k)[Sn,S_] of the form 


The inner summation Fy(n,k) = Y: 


ag = ((k+2)(n—k—-1)(nt+k+2)S7+(...)Se+(...), 
(n+k+2)SpSp+(..)Sn+(..)SR#(..)5 
(n+2)3(n+k+2)82+(...)Snt(..)Se+(..)) 


for the sum Fy(n,k). Now closure properties algorithms can turn ay, a2 and a de- 


2 
scription for Cres into a D-finite description a for the summand of F(n), and 


from this description a, a summation algorithm finds a messy fourth order recurrence 
for the sum F(n). 

Why does it not find the nice second order recurrence that we found in Exam- 
ple 15.4.7? The reason is that we lost a relation when we combined the D-finite 
descriptions a, and ay to a D-finite description for F\(n) + Fa(n,k). The usual algo- 
rithm for doing this yields in fact a D-finite description that is valid for every linear 
combination OF, (n) + BFy(n,k) (a@,B € Q), but it so happens that in the present 
example there is an additional annihilating operator for & = B = 1. It has the form 
(...)Se + (...)Sn + (...). If we apply creative telescoping to the ideal obtained by 
adding this operator to a, the result is the recurrence from Example 15.4.7, and the 
computation also goes much faster. See Example 15.5.3 for how such extra operators 
can be found. 


15.4.3.4 Nested sums and integrals 


Since the sum/integral over a holonomic function with respect to a dis- 
crete/continuous variable is a holonomic function in the remaining variables, it is 
clear that nested sums or integrals, and even mixed expressions involving summa- 
tions as well as integrations, can be handled by applying creative telescoping, possi- 
bly in combination with algorithms for closure properties, repeatedly. 

An alternative to repeated creative telescoping is multivariate creative telescop- 
ing, which is defined as follows. For notational simplicity, we formulate the definition 
only for the differential case. LetA:=C(x1,...,%u,Y1,---,Jv)[Dx,,---,Dx,,91,---, 9] 
and let a CA be a D-finite ideal. A telescoper for a with respect to the vari- 
ables x1,...,X, is an operator P € C(y1,...,¥y)[01,---,] \ {0} such that there exist 
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O1,...,0, € A with 


P Dy, Q1 Dy, Q2 Ban Dy, Qu a. 


Applications of this version of creative telescoping include the following. 


Integrals. If the integral F = fy fd(x1,...,x,) has natural boundaries, then 
every telescoper for the integrand f is an annihilating operator for the inte- 
gral F. 


Residues. If f = Yj, i, Gi,,..iy (15 +> we ---x/v ig a multivariate Laurent 
series in the sense of Section 15.2.3, define the residue of f with respect to 
X1,...,X%y, as F = Pas bivign i =a 42. —1()1,---,)v)- Then every telescoper 


for f with respect to x,,...,x, 1S an annihilating operator for F. 


Constant Terms. From the previous remark it follows immediately that a 
telescoper for se aoe oe 'f is an annihilating operator for the constant term 


F = [x?...x0] f. More generally, when (m1,...,7u) € Z" is fixed then every 


- 1 - 1 é : vas 
telescoper for x, ars Xy (mut df with respect to x1,...,y, 1s an annihilat- 


ing operator for F = ie Side gy Sy goa 


Diagonals. [121, 35] The diagonal of a multivariate power series 


Ff Biysse5 = ) a ae A .-.xlu is defined as the univariate 
power series d(y) := )"_4 Ann,....ny'. Every telescoper for the transformed 
series PTE ACAE Pr is CED. with respect to x1,...,X,—1 iS an anni- 


hilating operator for d(y). 


Positive Parts. [33] If f(x1,...,x,) is a multivariate formal Laurent series 
in the sense of Section 15.2.3 whose support is contained in a half space 
H C Z" that contains the principal orthant N”, then every telescoper for 
TRC AEA CT pebhy =) with respect to y;,...,y, 18 an annihilating op- 


ae Ts Vee «128s 
erator for the positive part [pate le OF s- 


15.4.4 Implementations and current research topics 


The classical algorithms described in Section 15.4.1 were implemented in the 1990s 
and are now included in the standard libraries of many general purpose computer 
algebra systems. Hypergeometric single sums arising in combinatorics are usually 
no challenge for these implementations. 

The advanced summation technology described in Sections 15.4.2 and 15.4.3 is 
not yet as widely available. The only known implementation of ITx-theory is Schnei- 
der’s Mathematica package Sigma [159]. For holonomic functions, there is Chyzak’s 
package mgfun [60] for Maple (part of the Algolib library [8]), and a Mathemat- 
ica package by Koutschan [110, 112]. For holonomic functions in a single variable, 
see [154, 129, 100]. 
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There are some even more advanced summation algorithms. For example, there 
are extensions of the ITZ-theory towards sums involving special functions [157], un- 
specified summands [104, 103], or radical expressions [105], and the holonomic 
systems approach has been generalized to Abel type sums [128], sums involving 
Bernoulli numbers [59], Stirling numbers [95] and other non-holonomic quanti- 
ties [62]. 

Most of the research work during the first decade of the 21st century was on ex- 
tending the applicability of the classical summation algorithms to larger and larger 
classes. In recent years, the focus is moving more towards complexity analysis and 
the construction of more efficient algorithms. First results in this direction are var- 
ious bounds on the sizes of telescopers [31, 55, 56] as well as alternative creative 
telescoping algorithms that construct telescopers without constructing correspond- 
ing certificates [38, 32, 58]. Since certificates tend to be much larger than the corre- 
sponding telescopers, algorithms that avoid their calculation have a good chance to 
be much more efficient. Note that for many summation and integration problems the 
certificate is not needed. 


15.5 The guess-and-prove paradigm 


The algorithms discussed in this chapter are useful for delegating boring and tedious 
tasks that often arise in the context of research in combinatorics to a machine. Even 
if a sum is simple enough that we may have a chance to simplify it by hand using 
traditional paper and pencil calculations, it may be a good idea to let the computer do 
it, and to devote our own precious time and energy to more important matters. Many 
applications of computer algebra are of this sort. 

With the algorithms discussed in this chapter it is sometimes also possible to 
solve cutting edge research problems in combinatorics that are not accessible by any 
other means. It is however naive to expect that such problems can simply be solved 
by entering a command into a computer algebra system and waiting a minute or 
two for the solution. More realistically, in order to obtain nontrivial results using 
computer algebra, it is necessary to design an individual argument that reduces the 
problem at hand to one or more computational problems that can then be solved by 
the computer, possibly using enormous time and memory resources. 

It is thus acommon misconception that computer algebra would eventually make 
human creativity obsolete. Instead, the creativity that would otherwise be used to 
come up with a proof in the traditional way will now be needed to come up with a 
logical reduction of the problem under consideration to subproblems to which com- 
puter algebra techniques are applicable. Obviously, there is no systematic way of 
breaking a research problem into computable pieces, but a general guideline that of- 
ten leads to success is to proceed along the following three steps: (1) automated gen- 
eration of data, (2) automated extraction of conjectures from the data (“guessing”), 
(3) automated proof of the guessed conjectures. 
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Of course, a guessed conjecture may be false (although this virtually never hap- 
pens in practice), and in this case, step (3) cannot succeed. When this happens, try 
again with some more data. Increasing the amount of data supplied to the guessing 
method will make it harder for false conjectures to show up in the output, so that 
eventually only true (and provable) conjectures will remain. 

We illustrate the guess-and-prove paradigm on a collection of examples, starting 
with two artificial toy examples. 


Example 15.5.1 (Solving Transcendental Equations) Suppose we want to find a 
real number solution, other than x = 1, of the transcendental equation 


28(x — 1)log(3x—1) + (7x+2)log(2x? —2x+1) =0. 


No algorithms are known for solving such equations directly. However, we can pro- 
ceed as follows. 


I. (generate data) Using a numerical algorithm, we can find that there is a solu- 
tion x with x © 0.5714285714. 


2. (guess) Using lattice reduction (Section 15.2.1) or table lookup [169], we can 
find that this number is close to x = 4/7. 


3. (prove) Simplifying the expression obtained by plugging the guessed solution 
x = 4/7 into the equation, we can check that it is indeed a solution. 


Example 15.5.2 (Nonlinear Recurrences) Let f(n) be defined recursively by 
f(l) =9, f(2) =21, f(3) = 136, and 


(4n? — 3) f(n +1) f(n—1) = (4n? — 19) f (n)? + 108n* — 106n? + 19. 


Suppose we want to know whether f(n) is a polynomial sequence. No algorithms are 
known for deciding this directly. However, we can proceed as follows. 


1. (generate data) Use the recurrence to compute f(n) forn = 1,...,20, say. 


2. (guess) Compute the interpolating polynomial of this data. It is 2x* — 3x? +1. 
The fact that the degree is significantly smaller than the number of interpola- 
tion points is already a good indication that this polynomial is a good guess. 


3. (prove) For getting a rigorous proof, plug the polynomial into the equation 
and check that both sides are identical after simplification. This proves that we 
have indeed f(n) = 2n* —3n? +1 for alln EN. 


More generally, instead of finding a polynomial expression by interpolating data, 
we can also compute recurrence equations that match a given finite array of data. 
This technique enjoys a great popularity in experimental mathematics and it is of- 
ten used as a synonym of “guessing,” although it should better be only viewed as 
a particular technique for solving a particular type of guessing problems. The idea 
is easily explained. Suppose we know the first N + 1 terms co,...,cy of an infinite 
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sequence (c,,)*°_) and we are interested in operators L € K[n][S,,] that possibly anni- 
hilate (cy) . Since most terms of the sequence are unknown, there is no chance to 
ensure that a particular operator does annihilate the whole sequence, but we can use 
as anecessary condition that any operator that is valid for all terms must in particular 
be valid for the finitely many terms we know explicitly. We can therefore make an 
ansatz with undetermined coefficients ¢; ; for an operator 


L= (€o0+l01n+ loan" +++ + loan") 
+ (£10 +2, 1n +4, 2n? arses +1 4n")Sn 


+ (0,9 +£,1n + bon? +++ +l,an")S? 


of order r and degree d, for some suitably chosen r,d € N, and require that the first 
N-—r terms of the sequence L- (c,)*°_») be zero. This gives a system of N — r homoge- 
neous linear equations for the (r+ 1)(d +1) many unknown coefficients ¢; ;, which 
we can solve. Typically, when N —r > (r+ 1)(d+1), such a system won’t have any 
nontrivial solution, and on the other hand, every true operator must belong to the 
solution space. Therefore, if there is no solution, this proves that there is no operator 
of order < r and degree < d that annihilates (cn);_-0> and if there is a solution, then 
it is reasonable to “guess” that the corresponding operator will annihilate the entire 
sequence. The confidence of the guess can be increased by increasing the difference 
N—r—(r+1)(d+1), but without using further information about the problem from 
which the sequence originates no finite amount of data will ever provide a rigorous 
proof that the operator is correct. 

The idea to make an ansatz for an equation and to match it against known data is 
very versatile. It can also be used to search for linear or nonlinear algebraic or differ- 
ential equations satisfied by a formal power series of which the first terms are known, 
and even multivariate recurrence equations satisfied by sequences depending on sev- 
eral indices or multivariate functional equations satisfied by power series in several 
variables. For the most common cases of guessing univariate recurrence or differen- 
tial operators efficient algorithms are known whose performance is so much better 
than the naive ansatz sketched above that in applications it is not unusual that the 
computation time needed for guessing is negligible compared to the time needed for 
generating data. For details on such guessing algorithms and their implementations 
for various computer algebra systems, see [24, 154, 115, 97, 37, 83, 99]. 

We conclude this chapter with three non-trivial applications of the guess-and- 
prove paradigm. 


Example 15.5.3 (Annihilator Completion) Jn Example 15.4.10 we showed that the 
bivariate sequence 


(— Ly 


ny k 
Pk) = 3+ Loom ari 
=e PO IGD 
has an annihilating ideal of the form 


a=((...)S+(...)Se+(-), 
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(SSE GS CaS Ga 
(...)S2+(..)Sa tC.) +(.-)) © Qn, k)[Sn, Se]. 


This is, however, not the complete annihilator. There are annihilating operators of 
f(n,k) that do not belong to a. 

To find such an operator, use the known operators (or the original sum repre- 
sentation) to compute the terms f(n,k) for, say, n,k =1,...,20, and use this data 
to guess an operator involving only terms SS! that are not divisible by the leading 
terms of the generators of a. This gives the conjectured operator 


L=(n+1)?S,+2(k—n)(k + 1)S_ — (n? — 2kn + 2k? +2k +1). 


By construction, we then have (L- f)(n,k) =0 forn,k =0,...,19. 

To prove that L really annihilates f, use D-finite closure properties (Sec- 
tion 15.4.3) to compute an annihilating ideal b for g:= L-f. We find b = ((n+ 
2)Sn—(n+1),S;—1). By checking that g(1,1) = 0, the generators of 6 imply that 
g(n,k) =0 for all n,k, thus proving that L annihilates f. 


Example 15.5.4 (Restricted Lattice Walks II) Let f(n,i,j) be the number of 
Kreweras walks with endpoint (i,j), introduced in Example 15.3.2 above. Krew- 
eras [117] showed that the generating function F(t,x,y) = Lei jcot (n,i, j)x'ylt" 
is algebraic. This result can also be obtained using the guess-and-proof paradigm, 
after making some preparatory observations. 

From the combinatorial definition we have the recurrence equation 


f(n+1ij) = fit hj+1)+fai-Lijt+faij-Y) 


for n> 0, the boundary conditions f(0,0,0) = 1 and f(n,i,—1) = f(n,-1,j) = 
0 for all n,i, j, and the symmetry f(n,i,j) = f(n, j,i). For the generating function 
F(t,x,y), these facts imply the functional equation 


((x+y+2°y") - 2) F(t.x,y) =xF(t,x,0)+yF(t,y,0) — 


IfY (x,t) € Q(x)|[t]] is a root of the polynomial in front of F (t,x,y), then this equation 
implies that the series U = F (t,x,0) is a solution of the functional equation 


U(t,x) = “Y(t. - *¥(t,2)U(Y (tx). 

It is not difficult to see that F (t,x,0) is in fact the only power series solution of this 
equation. Therefore, if we can show that the equation has an algebraic power series 
solution U, then this solution must be identical to F (t,x,0), and thus F (t,x,0) must 
be algebraic. It then follows from closure properties for algebraic functions that also 
F(t,x,y) is algebraic. 

To see that the functional equation for U has an algebraic power series solution, 
first compute a number of terms of F (t,x,0) using the recurrence for f(n,i,j) stated 
above and the fact {t"|F (t,x,0) =Y"_o f (n,i,0)x!. Use this data to guess an algebraic 
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equation possibly satisfied by F (t ,x,0). Check that this algebraic equation does have 
indeed a power series solution, and prove, using closure properties for algebraic 
functions, that this power series solution satisfies the functional equation. 

A detailed description of all the steps of this computation can be found in [39]. A 
different method, also following the guess-and-proof paradigm, is explained in [106, 
101]. This latter method can be used to derive and prove formulas for the counting 
sequence f(n,0,0) of walks returning to the origin. 


Example 15.5.5 (Determinant Identities) Zeilberger [194] points out that an ap- 
proach by Mills et al. [135] for proving certain types of determinant identities can 
be executed in a guess-and-prove fashion. The q-TSPP conjecture [113] was proven 
in this way, as well as some other conjectured determinant identities [114]. 

The approach is applicable to identities for n x n determinants, where n is a 
variable, and the (i, j)-th entry of the n-th matrix is given by a bivariate holonomic 
sequence (di j)ij—1 so that forn = 1,2,3,4,... we get the determinants 

a1 42 413 414 

a1 412 413 

421 422 423 424 
’ 42,1 42.2 423), ’ 
431 432 433 434 

431 432 433 

a41 442 443 a4, 


Let it be conjectured that the n-th determinant in this sequence admits a closed form 
representation of the form |];_, bx, where by is some holonomic sequence with by #0 
for all k. 

Write A, for the nth determinant in the sequence, and let AW be the cofactors 


(/) 


appearing in the Laplace expansion along the nth row, so that Ay = i=l Gn, jAi 


for all n. It is then easy to see that for the sequence Cp, = AY /A, we have the two 
identities 


n 
Y? ai,jen,j =0 (i=1,...,n—1), and Can = 1. 
j=l 


It is also easy to see that the sequence Cy; is uniquely determined by these two 
identities. 

Now compute the values for Cy,j for some 1 <n,j <N, for some sufficiently 
large N, and use this data to guess a holonomic description of Cy,j. In theory, there 
is no reason why the sequence (Cnj)n j=l should be holonomic, and if it isn’t, the 
approach fails. But if a (guessed) holonomic description is discovered, then it can 
be decided using a summation algorithm (Section 15.4.3) whether every solution 
€n,j Of the guessed holonomic system satisfies the two identities stated above, which, 
by uniqueness, implies that Cn,j = €n,j, and hence that cy; is holonomic. Finally, to 
conclude the proof, observe that the conjectured determinant identity is equivalent 
to by = Vin Qn,jCn,j, Which, using the guessed-and-proved holonomic description 
of Cn,j, can be proven using again a summation algorithm. 
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